From patchwork Thu Mar 30 06:30:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B380C6FD1D for ; Thu, 30 Mar 2023 06:31:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230048AbjC3GbJ (ORCPT ); Thu, 30 Mar 2023 02:31:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229620AbjC3GbH (ORCPT ); Thu, 30 Mar 2023 02:31:07 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA1564ED2 for ; Wed, 29 Mar 2023 23:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=7NM/Q3gGl4/n4XP7OUJDq9qpA/aF9eNuVJD/ObDGwrU=; b=Ka7txHJ5/HzlcmcyjOGC537s92 y6x738E6OkS/U1VbPAuv9YiN0ydL87XNHG4hQg01B8kXCvXgBErw2gjbaU9iIk/hltca4Ns9aKYio FE/R406wGUOVdXcv+OsevaI76DQ5V79eikJ1FV9+DWP9/5BAheN5zKmOD7662c2Nj+wUplYhyZRMZ Kzv+6zuXWBgWBT8GpB9HF7KB9fVd0+nIwcbj0fbtxF1s+wMAVTdMAxNeDOMsuBLxU57b2dvE5Rgbt 2IJ6m9uoC2w/7p3yhXyYWikjMxtt2hRXvccH9eOKRjotHbmN7+E4sSFzmbDedAp/AhdcHFUXp23+1 AWO9PmUw==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloO-002laQ-0j; Thu, 30 Mar 2023 06:31:04 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 01/21] btrfs: mark extent_buffer_under_io static Date: Thu, 30 Mar 2023 15:30:39 +0900 Message-Id: <20230330063059.1574380-2-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org extent_buffer_under_io is only used in extent_io.c, so mark it static. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/extent_io.c | 2 +- fs/btrfs/extent_io.h | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 1221f699ffc596..302af9b01bda2a 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -3420,7 +3420,7 @@ static void __free_extent_buffer(struct extent_buffer *eb) kmem_cache_free(extent_buffer_cache, eb); } -int extent_buffer_under_io(const struct extent_buffer *eb) +static int extent_buffer_under_io(const struct extent_buffer *eb) { return (atomic_read(&eb->io_pages) || test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) || diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 4341ad978fb8e4..342412d37a7b4b 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -265,7 +265,6 @@ void extent_buffer_bitmap_clear(const struct extent_buffer *eb, bool set_extent_buffer_dirty(struct extent_buffer *eb); void set_extent_buffer_uptodate(struct extent_buffer *eb); void clear_extent_buffer_uptodate(struct extent_buffer *eb); -int extent_buffer_under_io(const struct extent_buffer *eb); void extent_range_clear_dirty_for_io(struct inode *inode, u64 start, u64 end); void extent_range_redirty_for_io(struct inode *inode, u64 start, u64 end); void extent_clear_unlock_delalloc(struct btrfs_inode *inode, u64 start, u64 end, From patchwork Thu Mar 30 06:30:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D549C761AF for ; Thu, 30 Mar 2023 06:31:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229888AbjC3GbK (ORCPT ); Thu, 30 Mar 2023 02:31:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229529AbjC3GbI (ORCPT ); Thu, 30 Mar 2023 02:31:08 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 44CD94C3B for ; Wed, 29 Mar 2023 23:31:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=2tOevcsmIsATWobeokHa9OhlXHzX5+w8N9oqbV5Bd3I=; b=HxCIE34meTeJYkADraSzCzSbdK I0Y6Y8y1bfsNxwAx1e5RpRPsE3tgOiJ6N4kaZUmXwGyi9J32zD3ZiIB7r/Hlxtu51e9diKj7nPWBG 9RB9/8n3lWPtzu+6CF7oYaleM8d7ChKXDSe7SX2CAE4mcebvxfa0ecQW7fXI3io23HGqnYR5wkmbv pH5+6KmEnjs1QQQ9vCt1Z7d0M1K2rPNeVVHyB34VuXfGbG3RFYRVBNjpBCf++Xre4Y+8xW0pP6ia6 lhVVzOdGTkdAKwIceWsNKhBLs/wDPHf9wxbMDrzTr9E6sINUR7hk7FTt7rGCgR13kJxirnK1AZUbh UmvABpkQ==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloQ-002lam-1q; Thu, 30 Mar 2023 06:31:06 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Qu Wenruo , Qu Wenruo Subject: [PATCH 02/21] btrfs: fix sub-page error bit in end_bio_subpage_eb_writepage Date: Thu, 30 Mar 2023 15:30:40 +0900 Message-Id: <20230330063059.1574380-3-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Call btrfs_page_clear_uptodate instead of ClearPageUptodate to properly manage the uptodate bit for the subpage case. Reported-by: Qu Wenruo Signed-off-by: Christoph Hellwig Reviewed-by: Qu Wenruo --- fs/btrfs/extent_io.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 302af9b01bda2a..2bc141b3f3bc4b 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1914,7 +1914,8 @@ static void end_bio_subpage_eb_writepage(struct btrfs_bio *bbio) if (bio->bi_status || test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags)) { - ClearPageUptodate(page); + btrfs_page_clear_uptodate(fs_info, page, + eb->start, eb->len); set_btree_ioerr(page, eb); } From patchwork Thu Mar 30 06:30:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86E27C74A5B for ; Thu, 30 Mar 2023 06:31:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230252AbjC3GbO (ORCPT ); Thu, 30 Mar 2023 02:31:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230138AbjC3GbN (ORCPT ); Thu, 30 Mar 2023 02:31:13 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FF6C4EEA for ; Wed, 29 Mar 2023 23:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=83c77I+4g+T4k/fr/P2gdwBFWJSyzDGdWJK9Ru6yn7g=; b=NdWIWpOsxH3zfcmf4G7YgVrw6Q qA3P3ReBefSJZV8fTjTHt7ORGZEwUQ04BHeo5l2ODGNu4CvjF7IfqhHzGd4KKficRTtxM8yD3L9NZ Gs2H31KxeUcVsMJAatTHE0NgIFYqjX1yja8bThUceuMMSToWvywMVjhrHwdzTf7DydjkU6+BXuNRj olFICeYmhrwK5CXCDdfaRc+Ii2391g37xjqohHCbXtJlt+/xrJxxLixwCVTiphp+dkUecbocsAEey IF7E+nwAm/fu5hSCjedIO2vxEJ+ueSTXGuiZjjEP0kIelrXDN1fZP6BoacUOkzQSlcacOBcnIk07X 7NdKsVTw==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloT-002lb2-0R; Thu, 30 Mar 2023 06:31:09 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 03/21] btrfs: move setting the buffer uptodate out of validate_extent_buffer Date: Thu, 30 Mar 2023 15:30:41 +0900 Message-Id: <20230330063059.1574380-4-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Setting the buffer uptodate in a function that is named as a validation helper is a it confusing. Move the call from validate_extent_buffer to the one of its two callers that didn't already have a duplicate call to set_extent_buffer_uptodate. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/disk-io.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 3f57c41f41bf5f..20043c28c992dd 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -590,9 +590,7 @@ static int validate_extent_buffer(struct extent_buffer *eb, if (found_level > 0 && btrfs_check_node(eb)) ret = -EIO; - if (!ret) - set_extent_buffer_uptodate(eb); - else + if (ret) btrfs_err(fs_info, "read time tree block corruption detected on logical %llu mirror %u", eb->start, eb->read_mirror); @@ -684,6 +682,8 @@ int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio, goto err; } ret = validate_extent_buffer(eb, &bbio->parent_check); + if (!ret) + set_extent_buffer_uptodate(eb); err: if (ret) { /* From patchwork Thu Mar 30 06:30:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6E41C761AF for ; Thu, 30 Mar 2023 06:31:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230213AbjC3GbP (ORCPT ); Thu, 30 Mar 2023 02:31:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230248AbjC3GbO (ORCPT ); Thu, 30 Mar 2023 02:31:14 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7053555AA for ; Wed, 29 Mar 2023 23:31:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=udLfmB+iwQMzzYAyXcPJgoXTpvOZqr0rQ66urMC6wRk=; b=n3RcTHQXrtBFN5p4cwtreHwAbV LtyALRReE016iP5l4FxUr8qRnBfepwgT/0aYyRZ5u4dmLQE+txBQu3aKF2SyVHZcqwmafUHTFEbOy VN35vtfNbufCvcqP//1VKaDhgNhUD/nNCxBSSvmYQuyBTXoqQABmfArzcvKaFolvtTNCcR3/kKghL qGex2XPigDkNrVI5R+DHVZjZdfcNV/qA17R5V271Hi85hx88c/4TD1wVYdECeTTh1SHpF1JuJDtgX 1lHdLFz1MZdfcxPcNGID7sxwcUZzUgeejHYrvxx6+bPatsBMxYSq8G5agmU2acymMXZSCUhIMji6e zU/EhlCA==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloU-002lbR-39; Thu, 30 Mar 2023 06:31:11 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 04/21] btrfs: merge verify_parent_transid and btrfs_buffer_uptodate Date: Thu, 30 Mar 2023 15:30:42 +0900 Message-Id: <20230330063059.1574380-5-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org verify_parent_transid is only called by btrfs_buffer_uptodate, which confusingly inverts the return value. Merge the two functions and reflow the parent_transid so that error handling is in a branch. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/disk-io.c | 46 +++++++++++++++------------------------------- 1 file changed, 15 insertions(+), 31 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 20043c28c992dd..cf8d68f29ca850 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -110,32 +110,33 @@ static void csum_tree_block(struct extent_buffer *buf, u8 *result) * detect blocks that either didn't get written at all or got written * in the wrong place. */ -static int verify_parent_transid(struct extent_io_tree *io_tree, - struct extent_buffer *eb, u64 parent_transid, - int atomic) +int btrfs_buffer_uptodate(struct extent_buffer *eb, u64 parent_transid, + int atomic) { + struct inode *btree_inode = eb->pages[0]->mapping->host; + struct extent_io_tree *io_tree = &BTRFS_I(btree_inode)->io_tree; struct extent_state *cached_state = NULL; - int ret; + int ret = 1; - if (!parent_transid || btrfs_header_generation(eb) == parent_transid) + if (!extent_buffer_uptodate(eb)) return 0; + if (!parent_transid || btrfs_header_generation(eb) == parent_transid) + return 1; + if (atomic) return -EAGAIN; lock_extent(io_tree, eb->start, eb->start + eb->len - 1, &cached_state); - if (extent_buffer_uptodate(eb) && - btrfs_header_generation(eb) == parent_transid) { - ret = 0; - goto out; - } - btrfs_err_rl(eb->fs_info, + if (!extent_buffer_uptodate(eb) || + btrfs_header_generation(eb) != parent_transid) { + btrfs_err_rl(eb->fs_info, "parent transid verify failed on logical %llu mirror %u wanted %llu found %llu", eb->start, eb->read_mirror, parent_transid, btrfs_header_generation(eb)); - ret = 1; - clear_extent_buffer_uptodate(eb); -out: + clear_extent_buffer_uptodate(eb); + ret = 0; + } unlock_extent(io_tree, eb->start, eb->start + eb->len - 1, &cached_state); return ret; @@ -4630,23 +4631,6 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) btrfs_close_devices(fs_info->fs_devices); } -int btrfs_buffer_uptodate(struct extent_buffer *buf, u64 parent_transid, - int atomic) -{ - int ret; - struct inode *btree_inode = buf->pages[0]->mapping->host; - - ret = extent_buffer_uptodate(buf); - if (!ret) - return ret; - - ret = verify_parent_transid(&BTRFS_I(btree_inode)->io_tree, buf, - parent_transid, atomic); - if (ret == -EAGAIN) - return ret; - return !ret; -} - void btrfs_mark_buffer_dirty(struct extent_buffer *buf) { struct btrfs_fs_info *fs_info = buf->fs_info; From patchwork Thu Mar 30 06:30:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C509C6FD1D for ; Thu, 30 Mar 2023 06:31:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230254AbjC3GbQ (ORCPT ); Thu, 30 Mar 2023 02:31:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230138AbjC3GbP (ORCPT ); Thu, 30 Mar 2023 02:31:15 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 824CA61AA for ; Wed, 29 Mar 2023 23:31:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Z2Bi3BEmkBO8Hp3XDzcHafaV+JpO35yHTdK9JeNr1y0=; b=Wp5sgoVnejuFemQQJKo67nvuNF lmfrHacx1/STanv8SbV3/+SCNJw4GFCmPDK9NlXFwoyQ7baGbzguTAoymzL2/NogJpK6E7iGNv5rp agUKqt9QwY3u7Yc2WrBhIhgzuVEtwCJXgtxCCzxoc/0VcDMI9ixeuOEepEz/tQnGmo8mNrLrlsiMB DOxdCOiEoGOuhUbw/L9TEFM8Qc1uFJBgOFGHHIrqQU0E+jP63ZtZFN2pe3ltiSz500/L+BND52pyT vFhuidBQpMaigXYtreTHROArgc+zh/at7y2BU1jHm4lz7Dvw9ekFMrMnC/ePir7sD/Slw6PfUoMup Lzh8NuvQ==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloX-002lc7-06; Thu, 30 Mar 2023 06:31:13 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 05/21] btrfs: always read the entire extent_buffer Date: Thu, 30 Mar 2023 15:30:43 +0900 Message-Id: <20230330063059.1574380-6-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently read_extent_buffer_pages skips pages that are already uptodate when reading in an extent_buffer. While this reduces the amount of data read, it increases the number of I/O operations as we now need to do multiple I/Os when reading an extent buffer with one or more uptodate pages in the middle of it. On any modern storage device, be that hard drives or SSDs this actually decreases I/O performance. Fortunately this case is pretty rare as the pages are always initially read together and then aged the same way. Besides simplifying the code a bit as-is this will allow for major simplifications to the I/O completion handler later on. Note that the case where all pages are uptodate is still handled by an optimized fast path that does not read any data from disk. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/extent_io.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 2bc141b3f3bc4b..e7a0ef6d70bfe1 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4314,7 +4314,6 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, int locked_pages = 0; int all_uptodate = 1; int num_pages; - unsigned long num_reads = 0; struct btrfs_bio_ctrl bio_ctrl = { .opf = REQ_OP_READ, .mirror_num = mirror_num, @@ -4360,10 +4359,8 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, */ for (i = 0; i < num_pages; i++) { page = eb->pages[i]; - if (!PageUptodate(page)) { - num_reads++; + if (!PageUptodate(page)) all_uptodate = 0; - } } if (all_uptodate) { @@ -4373,7 +4370,7 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); eb->read_mirror = 0; - atomic_set(&eb->io_pages, num_reads); + atomic_set(&eb->io_pages, num_pages); /* * It is possible for release_folio to clear the TREE_REF bit before we * set io_pages. See check_buffer_tree_ref for a more detailed comment. @@ -4383,13 +4380,9 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, for (i = 0; i < num_pages; i++) { page = eb->pages[i]; - if (!PageUptodate(page)) { - ClearPageError(page); - submit_extent_page(&bio_ctrl, page_offset(page), page, - PAGE_SIZE, 0); - } else { - unlock_page(page); - } + ClearPageError(page); + submit_extent_page(&bio_ctrl, page_offset(page), page, + PAGE_SIZE, 0); } submit_one_bio(&bio_ctrl); From patchwork Thu Mar 30 06:30:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7D63C761AF for ; Thu, 30 Mar 2023 06:31:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230248AbjC3GbS (ORCPT ); Thu, 30 Mar 2023 02:31:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229597AbjC3GbR (ORCPT ); Thu, 30 Mar 2023 02:31:17 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF7E04EFC for ; Wed, 29 Mar 2023 23:31:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=lrY+PNA3btVfKte69b4KeQJJK32O3tPdJYJ3+Hjc2AU=; b=kKMpeQEsRSWJHpX4Z/TE2pTB5e RoQSEbglg2aNekgDwJ9tXQ32AMC+4j4nlVUvjgdaQOJ2wUxyLD1JqF9A6z8qh/s+7w7Ijkb7l0ar7 PgLpheFMDMCTpeuh+7gfEbVk4xlH8960bBgfHr7DPoG98ycehpcdypqE9/He2ULnlrJnBewCEXYOD FyfruktBlrUs/l4ViU+yM5JfT9S1HFP5vTFpuhBPQ+pRegMOvDzbUyDH2oBQMdK8KHbHWvUyVWN4j 0K3YtTn9TrUQ5S0n6q+RmmKMQdJpQedvW1LFiA7yP+L73o9Feo8gem5HXi2igvVbqdGUs9YJort2I dbufdocQ==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloZ-002lcO-1E; Thu, 30 Mar 2023 06:31:15 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 06/21] btrfs: don't use btrfs_bio_ctrl for extent buffer reading Date: Thu, 30 Mar 2023 15:30:44 +0900 Message-Id: <20230330063059.1574380-7-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The btrfs_bio_ctrl machinery is overkill for reading extent_buffers as we always operate on PAGE SIZE chunks (or one smaller one for the subpage case) that are contigous and are guaranteed to fit into a single bio. Replace it with open coded btrfs_bio_alloc, __bio_add_page and btrfs_submit_bio calls in a helper function shared between the subpage and node size >= PAGE_SIZE cases. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/extent_io.c | 99 ++++++++++++++++---------------------------- 1 file changed, 36 insertions(+), 63 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index e7a0ef6d70bfe1..8e709b44fa57ec 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -98,22 +98,12 @@ void btrfs_extent_buffer_leak_debug_check(struct btrfs_fs_info *fs_info) */ struct btrfs_bio_ctrl { struct btrfs_bio *bbio; - int mirror_num; enum btrfs_compression_type compress_type; u32 len_to_oe_boundary; blk_opf_t opf; btrfs_bio_end_io_t end_io_func; struct writeback_control *wbc; - /* - * This is for metadata read, to provide the extra needed verification - * info. This has to be provided for submit_one_bio(), as - * submit_one_bio() can submit a bio if it ends at stripe boundary. If - * no such parent_check is provided, the metadata can hit false alert at - * endio time. - */ - struct btrfs_tree_parent_check *parent_check; - /* * Tell writepage not to lock the state bits for this range, it still * does the unlocking. @@ -124,7 +114,6 @@ struct btrfs_bio_ctrl { static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl) { struct btrfs_bio *bbio = bio_ctrl->bbio; - int mirror_num = bio_ctrl->mirror_num; if (!bbio) return; @@ -132,25 +121,14 @@ static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl) /* Caller should ensure the bio has at least some range added */ ASSERT(bbio->bio.bi_iter.bi_size); - if (!is_data_inode(&bbio->inode->vfs_inode)) { - if (btrfs_op(&bbio->bio) != BTRFS_MAP_WRITE) { - /* - * For metadata read, we should have the parent_check, - * and copy it to bbio for metadata verification. - */ - ASSERT(bio_ctrl->parent_check); - memcpy(&bbio->parent_check, - bio_ctrl->parent_check, - sizeof(struct btrfs_tree_parent_check)); - } + if (!is_data_inode(&bbio->inode->vfs_inode)) bbio->bio.bi_opf |= REQ_META; - } if (btrfs_op(&bbio->bio) == BTRFS_MAP_READ && bio_ctrl->compress_type != BTRFS_COMPRESS_NONE) - btrfs_submit_compressed_read(bbio, mirror_num); + btrfs_submit_compressed_read(bbio, 0); else - btrfs_submit_bio(bbio, mirror_num); + btrfs_submit_bio(bbio, 0); /* The bbio is owned by the end_io handler now */ bio_ctrl->bbio = NULL; @@ -4242,6 +4220,36 @@ void set_extent_buffer_uptodate(struct extent_buffer *eb) } } +static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, + struct btrfs_tree_parent_check *check) +{ + int num_pages = num_extent_pages(eb), i; + struct btrfs_bio *bbio; + + clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); + eb->read_mirror = 0; + atomic_set(&eb->io_pages, num_pages); + check_buffer_tree_ref(eb); + + bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, + REQ_OP_READ | REQ_META, + BTRFS_I(eb->fs_info->btree_inode), + end_bio_extent_readpage, NULL); + bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; + bbio->file_offset = eb->start; + memcpy(&bbio->parent_check, check, sizeof(*check)); + if (eb->fs_info->nodesize < PAGE_SIZE) { + __bio_add_page(&bbio->bio, eb->pages[0], eb->len, + eb->start - page_offset(eb->pages[0])); + } else { + for (i = 0; i < num_pages; i++) { + ClearPageError(eb->pages[i]); + __bio_add_page(&bbio->bio, eb->pages[i], PAGE_SIZE, 0); + } + } + btrfs_submit_bio(bbio, mirror_num); +} + static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, int mirror_num, struct btrfs_tree_parent_check *check) @@ -4250,11 +4258,6 @@ static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, struct extent_io_tree *io_tree; struct page *page = eb->pages[0]; struct extent_state *cached_state = NULL; - struct btrfs_bio_ctrl bio_ctrl = { - .opf = REQ_OP_READ, - .mirror_num = mirror_num, - .parent_check = check, - }; int ret; ASSERT(!test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags)); @@ -4282,18 +4285,10 @@ static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, return 0; } - clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); - eb->read_mirror = 0; - atomic_set(&eb->io_pages, 1); - check_buffer_tree_ref(eb); - bio_ctrl.end_io_func = end_bio_extent_readpage; - btrfs_subpage_clear_error(fs_info, page, eb->start, eb->len); - btrfs_subpage_start_reader(fs_info, page, eb->start, eb->len); - submit_extent_page(&bio_ctrl, eb->start, page, eb->len, - eb->start - page_offset(page)); - submit_one_bio(&bio_ctrl); + + __read_extent_buffer_pages(eb, mirror_num, check); if (wait != WAIT_COMPLETE) { free_extent_state(cached_state); return 0; @@ -4314,11 +4309,6 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, int locked_pages = 0; int all_uptodate = 1; int num_pages; - struct btrfs_bio_ctrl bio_ctrl = { - .opf = REQ_OP_READ, - .mirror_num = mirror_num, - .parent_check = check, - }; if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) return 0; @@ -4368,24 +4358,7 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, goto unlock_exit; } - clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); - eb->read_mirror = 0; - atomic_set(&eb->io_pages, num_pages); - /* - * It is possible for release_folio to clear the TREE_REF bit before we - * set io_pages. See check_buffer_tree_ref for a more detailed comment. - */ - check_buffer_tree_ref(eb); - bio_ctrl.end_io_func = end_bio_extent_readpage; - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - - ClearPageError(page); - submit_extent_page(&bio_ctrl, page_offset(page), page, - PAGE_SIZE, 0); - } - - submit_one_bio(&bio_ctrl); + __read_extent_buffer_pages(eb, mirror_num, check); if (wait != WAIT_COMPLETE) return 0; From patchwork Thu Mar 30 06:30:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5082CC6FD1D for ; Thu, 30 Mar 2023 06:31:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230024AbjC3GbU (ORCPT ); Thu, 30 Mar 2023 02:31:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53658 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230009AbjC3GbT (ORCPT ); Thu, 30 Mar 2023 02:31:19 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2729C6A58 for ; Wed, 29 Mar 2023 23:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=af6CFNcx1Ql/QhkvTdRDFplAwzo/K8tHiN7hqfiYfCI=; b=aLOzuOVQCuBeTCvhkXxulfeptY JAY8DPsdYQ7NHOpGY7M2f7juoplGR3RiNYVUttC4s0Dhua/wJLW+XslONJFuUzNoaGIrjrHYn3MW0 gzX1wbdQd0NCO89CCdOsR9w9iC4FY7RGk37cvHmZ7FTfu6YDzcxA5oTpB2f9fEmZBPtXOfqUVHKTV EQRCueQAZrhVnhysro7AGt56UZa/Jg17S2W/ALWLAmFxEZcsUxG76ax+PNWdIJZpy90JfhzspDKPu pkK8K1xCyTE3G9x2+giYKAnjd7nuAuq80+/uxv3fOA5ZJB5qKkTnarnydYVMjcn3T6NWasOwA3IE5 Fv1DRrew==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlob-002lcv-2N; Thu, 30 Mar 2023 06:31:18 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 07/21] btrfs: remove the mirror_num argument to btrfs_submit_compressed_read Date: Thu, 30 Mar 2023 15:30:45 +0900 Message-Id: <20230330063059.1574380-8-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Given that read recovery for data I/O is handled in the storage layer, the mirror_num argument to btrfs_submit_compressed_read is always 0, so remove it. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/compression.c | 4 ++-- fs/btrfs/compression.h | 2 +- fs/btrfs/extent_io.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 44c4276741ceda..8ca152164c11bf 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -479,7 +479,7 @@ static noinline int add_ra_bio_pages(struct inode *inode, * After the compressed pages are read, we copy the bytes into the * bio we were passed and then call the bio end_io calls */ -void btrfs_submit_compressed_read(struct btrfs_bio *bbio, int mirror_num) +void btrfs_submit_compressed_read(struct btrfs_bio *bbio) { struct btrfs_inode *inode = bbio->inode; struct btrfs_fs_info *fs_info = inode->root->fs_info; @@ -545,7 +545,7 @@ void btrfs_submit_compressed_read(struct btrfs_bio *bbio, int mirror_num) if (memstall) psi_memstall_leave(&pflags); - btrfs_submit_bio(&cb->bbio, mirror_num); + btrfs_submit_bio(&cb->bbio, 0); return; out_free_compressed_pages: diff --git a/fs/btrfs/compression.h b/fs/btrfs/compression.h index 5d5146e72a860b..8ba8e62b096061 100644 --- a/fs/btrfs/compression.h +++ b/fs/btrfs/compression.h @@ -94,7 +94,7 @@ void btrfs_submit_compressed_write(struct btrfs_inode *inode, u64 start, blk_opf_t write_flags, struct cgroup_subsys_state *blkcg_css, bool writeback); -void btrfs_submit_compressed_read(struct btrfs_bio *bbio, int mirror_num); +void btrfs_submit_compressed_read(struct btrfs_bio *bbio); unsigned int btrfs_compress_str2level(unsigned int type, const char *str); diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 8e709b44fa57ec..4d412efe32c6b2 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -126,7 +126,7 @@ static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl) if (btrfs_op(&bbio->bio) == BTRFS_MAP_READ && bio_ctrl->compress_type != BTRFS_COMPRESS_NONE) - btrfs_submit_compressed_read(bbio, 0); + btrfs_submit_compressed_read(bbio); else btrfs_submit_bio(bbio, 0); From patchwork Thu Mar 30 06:30:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74489C6FD1D for ; Thu, 30 Mar 2023 06:31:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230256AbjC3GbY (ORCPT ); Thu, 30 Mar 2023 02:31:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230009AbjC3GbX (ORCPT ); Thu, 30 Mar 2023 02:31:23 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0471C658B for ; Wed, 29 Mar 2023 23:31:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=fDhBKKe3b6H38xPfPHLjEk+cBpTDfpw4OfzHDTgqprk=; b=HRL4A2qVEx1ueheBuoJQwsYNLT WbYFCanTVmI5yTWscXOU6skDlOXrRzMrymi6Iuyr30Pcs3KPp719JhMmXTR8H8QBUDW1JFZIPGAjX pIiLaIBO98LyXlqszuKThVEJPIKaIhLHP4fFmGy/Y7g4H071F+qZ0L+rufYdsm3PJHixjTTZ1OpL0 5T2j9suX5XsRQ0d831Gyc0p3PKpVcO+isMGtM4R3Tiv/Vdr5/1ysNoZWS/dW0LppUU5y/FXyKu5tP 7ZrIik7iBHd8/fhImHrbdqo0jR4Vkuc44+D0JT850LONHETaCdRhw/PTuu36EZ6iWwm4XjSLVFntD g52Fljzg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloe-002ldW-0x; Thu, 30 Mar 2023 06:31:20 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 08/21] btrfs: use a separate end_io handler for read_extent_buffer Date: Thu, 30 Mar 2023 15:30:46 +0900 Message-Id: <20230330063059.1574380-9-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we always use a single bio to read an extent_buffer, the buffer can be passed to the end_io handler as private data. This allows implementing a much simplified dedicated end I/O handler for metadata reads. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/disk-io.c | 105 +------------------------------------------ fs/btrfs/disk-io.h | 5 +-- fs/btrfs/extent_io.c | 80 +++++++++++++++------------------ 3 files changed, 41 insertions(+), 149 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index cf8d68f29ca850..4fd181f67925f9 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -485,8 +485,8 @@ static int check_tree_block_fsid(struct extent_buffer *eb) } /* Do basic extent buffer checks at read time */ -static int validate_extent_buffer(struct extent_buffer *eb, - struct btrfs_tree_parent_check *check) +int btrfs_validate_extent_buffer(struct extent_buffer *eb, + struct btrfs_tree_parent_check *check) { struct btrfs_fs_info *fs_info = eb->fs_info; u64 found_start; @@ -599,107 +599,6 @@ static int validate_extent_buffer(struct extent_buffer *eb, return ret; } -static int validate_subpage_buffer(struct page *page, u64 start, u64 end, - int mirror, struct btrfs_tree_parent_check *check) -{ - struct btrfs_fs_info *fs_info = btrfs_sb(page->mapping->host->i_sb); - struct extent_buffer *eb; - bool reads_done; - int ret = 0; - - ASSERT(check); - - /* - * We don't allow bio merge for subpage metadata read, so we should - * only get one eb for each endio hook. - */ - ASSERT(end == start + fs_info->nodesize - 1); - ASSERT(PagePrivate(page)); - - eb = find_extent_buffer(fs_info, start); - /* - * When we are reading one tree block, eb must have been inserted into - * the radix tree. If not, something is wrong. - */ - ASSERT(eb); - - reads_done = atomic_dec_and_test(&eb->io_pages); - /* Subpage read must finish in page read */ - ASSERT(reads_done); - - eb->read_mirror = mirror; - if (test_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags)) { - ret = -EIO; - goto err; - } - ret = validate_extent_buffer(eb, check); - if (ret < 0) - goto err; - - set_extent_buffer_uptodate(eb); - - free_extent_buffer(eb); - return ret; -err: - /* - * end_bio_extent_readpage decrements io_pages in case of error, - * make sure it has something to decrement. - */ - atomic_inc(&eb->io_pages); - clear_extent_buffer_uptodate(eb); - free_extent_buffer(eb); - return ret; -} - -int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio, - struct page *page, u64 start, u64 end, - int mirror) -{ - struct extent_buffer *eb; - int ret = 0; - int reads_done; - - ASSERT(page->private); - - if (btrfs_sb(page->mapping->host->i_sb)->nodesize < PAGE_SIZE) - return validate_subpage_buffer(page, start, end, mirror, - &bbio->parent_check); - - eb = (struct extent_buffer *)page->private; - - /* - * The pending IO might have been the only thing that kept this buffer - * in memory. Make sure we have a ref for all this other checks - */ - atomic_inc(&eb->refs); - - reads_done = atomic_dec_and_test(&eb->io_pages); - if (!reads_done) - goto err; - - eb->read_mirror = mirror; - if (test_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags)) { - ret = -EIO; - goto err; - } - ret = validate_extent_buffer(eb, &bbio->parent_check); - if (!ret) - set_extent_buffer_uptodate(eb); -err: - if (ret) { - /* - * our io error hook is going to dec the io pages - * again, we have to make sure it has something - * to decrement - */ - atomic_inc(&eb->io_pages); - clear_extent_buffer_uptodate(eb); - } - free_extent_buffer(eb); - - return ret; -} - #ifdef CONFIG_MIGRATION static int btree_migrate_folio(struct address_space *mapping, struct folio *dst, struct folio *src, enum migrate_mode mode) diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h index 4d577233011023..2923b5d7cfca0b 100644 --- a/fs/btrfs/disk-io.h +++ b/fs/btrfs/disk-io.h @@ -84,9 +84,8 @@ void btrfs_btree_balance_dirty(struct btrfs_fs_info *fs_info); void btrfs_btree_balance_dirty_nodelay(struct btrfs_fs_info *fs_info); void btrfs_drop_and_free_fs_root(struct btrfs_fs_info *fs_info, struct btrfs_root *root); -int btrfs_validate_metadata_buffer(struct btrfs_bio *bbio, - struct page *page, u64 start, u64 end, - int mirror); +int btrfs_validate_extent_buffer(struct extent_buffer *eb, + struct btrfs_tree_parent_check *check); #ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS struct btrfs_root *btrfs_alloc_dummy_root(struct btrfs_fs_info *fs_info); #endif diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 4d412efe32c6b2..5570f1050296c9 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -663,35 +663,6 @@ static void begin_page_read(struct btrfs_fs_info *fs_info, struct page *page) btrfs_subpage_start_reader(fs_info, page, page_offset(page), PAGE_SIZE); } -/* - * Find extent buffer for a givne bytenr. - * - * This is for end_bio_extent_readpage(), thus we can't do any unsafe locking - * in endio context. - */ -static struct extent_buffer *find_extent_buffer_readpage( - struct btrfs_fs_info *fs_info, struct page *page, u64 bytenr) -{ - struct extent_buffer *eb; - - /* - * For regular sectorsize, we can use page->private to grab extent - * buffer - */ - if (fs_info->nodesize >= PAGE_SIZE) { - ASSERT(PagePrivate(page) && page->private); - return (struct extent_buffer *)page->private; - } - - /* For subpage case, we need to lookup buffer radix tree */ - rcu_read_lock(); - eb = radix_tree_lookup(&fs_info->buffer_radix, - bytenr >> fs_info->sectorsize_bits); - rcu_read_unlock(); - ASSERT(eb); - return eb; -} - /* * after a readpage IO is done, we need to: * clear the uptodate bits on error @@ -713,7 +684,6 @@ static void end_bio_extent_readpage(struct btrfs_bio *bbio) * larger than UINT_MAX, u32 here is enough. */ u32 bio_offset = 0; - int mirror; struct bvec_iter_all iter_all; ASSERT(!bio_flagged(bio, BIO_CLONED)); @@ -753,11 +723,6 @@ static void end_bio_extent_readpage(struct btrfs_bio *bbio) end = start + bvec->bv_len - 1; len = bvec->bv_len; - mirror = bbio->mirror_num; - if (uptodate && !is_data_inode(inode) && - btrfs_validate_metadata_buffer(bbio, page, start, end, mirror)) - uptodate = false; - if (likely(uptodate)) { loff_t i_size = i_size_read(inode); pgoff_t end_index = i_size >> PAGE_SHIFT; @@ -778,13 +743,6 @@ static void end_bio_extent_readpage(struct btrfs_bio *bbio) zero_user_segment(page, zero_start, offset_in_page(end) + 1); } - } else if (!is_data_inode(inode)) { - struct extent_buffer *eb; - - eb = find_extent_buffer_readpage(fs_info, page, start); - set_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); - eb->read_mirror = mirror; - atomic_dec(&eb->io_pages); } /* Update page status and unlock. */ @@ -4220,6 +4178,42 @@ void set_extent_buffer_uptodate(struct extent_buffer *eb) } } +static void extent_buffer_read_end_io(struct btrfs_bio *bbio) +{ + struct extent_buffer *eb = bbio->private; + bool uptodate = !bbio->bio.bi_status; + struct bvec_iter_all iter_all; + struct bio_vec *bvec; + u32 bio_offset = 0; + + atomic_inc(&eb->refs); + eb->read_mirror = bbio->mirror_num; + + if (uptodate && + btrfs_validate_extent_buffer(eb, &bbio->parent_check) < 0) + uptodate = false; + + if (uptodate) { + set_extent_buffer_uptodate(eb); + } else { + clear_extent_buffer_uptodate(eb); + set_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); + } + + bio_for_each_segment_all(bvec, &bbio->bio, iter_all) { + atomic_dec(&eb->io_pages); + end_page_read(bvec->bv_page, uptodate, eb->start + bio_offset, + bvec->bv_len); + bio_offset += bvec->bv_len; + } + + unlock_extent(&bbio->inode->io_tree, eb->start, + eb->start + bio_offset - 1, NULL); + free_extent_buffer(eb); + + bio_put(&bbio->bio); +} + static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, struct btrfs_tree_parent_check *check) { @@ -4234,7 +4228,7 @@ static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, REQ_OP_READ | REQ_META, BTRFS_I(eb->fs_info->btree_inode), - end_bio_extent_readpage, NULL); + extent_buffer_read_end_io, eb); bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; bbio->file_offset = eb->start; memcpy(&bbio->parent_check, check, sizeof(*check)); From patchwork Thu Mar 30 06:30:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CADECC761AF for ; Thu, 30 Mar 2023 06:31:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230148AbjC3GbZ (ORCPT ); Thu, 30 Mar 2023 02:31:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230009AbjC3GbY (ORCPT ); Thu, 30 Mar 2023 02:31:24 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBB4261A4 for ; Wed, 29 Mar 2023 23:31:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=nkliL7/PV5NxqAfKFrVQuDY7ULiX7opspFI+9FZbZuE=; b=VYb/XAAfYMSdJmdXAKUUUycYBO g62qSJuNbA9VJy8q3jrCqfO2LeJbZ+7X3tSaz32T7nHyOYFOCZGjePF2v0YXO0sQ1flQpRkUl2Dbb Z+lybZcF2wzMz02jBYoBWstUYy52Q+i/W/B3T7UradF1dykY53VRoWKENLNZhtcq0/cJP+V5MJ7kU dlyPnA0Edwt4GtzekfW5873LhcutM48aeVORLFSVVjdUCpkoV13yo2pXCkN3CBBSvxaMdFIL3B41O S8f1ncF63bAeES5/aQkgeFWoSoKC7Jw0XtqnY1KVKbliSAFVV3dVLczCIIUftLw0mSdjjWxDGjogx 2MTcLVCg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlog-002le1-1R; Thu, 30 Mar 2023 06:31:22 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 09/21] btrfs: do not try to unlock the extent for non-subpage metadata reads Date: Thu, 30 Mar 2023 15:30:47 +0900 Message-Id: <20230330063059.1574380-10-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Only subpage metadata reads lock the extent. Don't try to unlock it and waste cycles in the extent tree lookup for PAGE_SIZE or larger metadata. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 5570f1050296c9..bc50163dd3b792 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4207,8 +4207,10 @@ static void extent_buffer_read_end_io(struct btrfs_bio *bbio) bio_offset += bvec->bv_len; } - unlock_extent(&bbio->inode->io_tree, eb->start, - eb->start + bio_offset - 1, NULL); + if (eb->fs_info->nodesize < PAGE_SIZE) { + unlock_extent(&bbio->inode->io_tree, eb->start, + eb->start + bio_offset - 1, NULL); + } free_extent_buffer(eb); bio_put(&bbio->bio); From patchwork Thu Mar 30 06:30:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0876DC6FD1D for ; Thu, 30 Mar 2023 06:31:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230258AbjC3Gb1 (ORCPT ); Thu, 30 Mar 2023 02:31:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230009AbjC3Gb0 (ORCPT ); Thu, 30 Mar 2023 02:31:26 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25E9D5BA0 for ; Wed, 29 Mar 2023 23:31:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=KJ8RdaCABGX208IoY0cIcWwADFt32og/FeiMSNahY1A=; b=xCTYZfY826eoLUTHDXfwi4TnKJ PSosbx+4iSEHmA3W0KlOSCZTWGC1C91E+DB8LsBC8aHOz7gN48MA29V2V1JAdqA7Qlh3SdpdznSbu uV4bG/0jGNIlolfVE9Xs/Aieu5Kno80wCsUnvYZNtsQEB9U44zGSN0eu0vSYmArslyFHAqFiqYrMl w9TwOntwXhMn6dPy+ff7DW5hP7AL6L9XyDh7U6S6+Ha73ceJ8PHq8l0eQJ1zrklf1NdFMi02JDBxa Ad+qzO7XI/vL8O7NqCwyI92qVWV1VvtP9THVmrIfd4OT/bLzniQpA9d4TITql4sP9h4YBsIyjEm/c HjNLUT1A==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloi-002leo-2G; Thu, 30 Mar 2023 06:31:25 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 10/21] btrfs: return bool from lock_extent_buffer_for_io Date: Thu, 30 Mar 2023 15:30:48 +0900 Message-Id: <20230330063059.1574380-11-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org lock_extent_buffer_for_io never returns a negative error value, so switch the return value to a simple bool. Also remove the noinline_for_stack annotation given that nothing in lock_extent_buffer_for_io or its callers is particularly stack hungry. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 39 ++++++++++++--------------------------- 1 file changed, 12 insertions(+), 27 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index bc50163dd3b792..08e4e53f42e8a7 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1628,18 +1628,17 @@ static void end_extent_buffer_writeback(struct extent_buffer *eb) * * May try to flush write bio if we can't get the lock. * - * Return 0 if the extent buffer doesn't need to be submitted. - * (E.g. the extent buffer is not dirty) - * Return >0 is the extent buffer is submitted to bio. - * Return <0 if something went wrong, no page is locked. + * Return %false if the extent buffer doesn't need to be submitted (e.g. the + * extent buffer is not dirty) + * Return %true is the extent buffer is submitted to bio. */ -static noinline_for_stack int lock_extent_buffer_for_io(struct extent_buffer *eb, - struct btrfs_bio_ctrl *bio_ctrl) +static bool lock_extent_buffer_for_io(struct extent_buffer *eb, + struct btrfs_bio_ctrl *bio_ctrl) { struct btrfs_fs_info *fs_info = eb->fs_info; int i, num_pages; int flush = 0; - int ret = 0; + bool ret = false; if (!btrfs_try_tree_write_lock(eb)) { submit_write_bio(bio_ctrl, 0); @@ -1650,7 +1649,7 @@ static noinline_for_stack int lock_extent_buffer_for_io(struct extent_buffer *eb if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) { btrfs_tree_unlock(eb); if (bio_ctrl->wbc->sync_mode != WB_SYNC_ALL) - return 0; + return false; if (!flush) { submit_write_bio(bio_ctrl, 0); flush = 1; @@ -1677,7 +1676,7 @@ static noinline_for_stack int lock_extent_buffer_for_io(struct extent_buffer *eb percpu_counter_add_batch(&fs_info->dirty_metadata_bytes, -eb->len, fs_info->dirty_metadata_batch); - ret = 1; + ret = true; } else { spin_unlock(&eb->refs_lock); } @@ -2011,7 +2010,6 @@ static int submit_eb_subpage(struct page *page, struct btrfs_bio_ctrl *bio_ctrl) u64 page_start = page_offset(page); int bit_start = 0; int sectors_per_node = fs_info->nodesize >> fs_info->sectorsize_bits; - int ret; /* Lock and write each dirty extent buffers in the range */ while (bit_start < fs_info->subpage_info->bitmap_nr_bits) { @@ -2057,25 +2055,13 @@ static int submit_eb_subpage(struct page *page, struct btrfs_bio_ctrl *bio_ctrl) if (!eb) continue; - ret = lock_extent_buffer_for_io(eb, bio_ctrl); - if (ret == 0) { - free_extent_buffer(eb); - continue; + if (lock_extent_buffer_for_io(eb, bio_ctrl)) { + write_one_subpage_eb(eb, bio_ctrl); + submitted++; } - if (ret < 0) { - free_extent_buffer(eb); - goto cleanup; - } - write_one_subpage_eb(eb, bio_ctrl); free_extent_buffer(eb); - submitted++; } return submitted; - -cleanup: - /* We hit error, end bio for the submitted extent buffers */ - submit_write_bio(bio_ctrl, ret); - return ret; } /* @@ -2154,8 +2140,7 @@ static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, *eb_context = eb; - ret = lock_extent_buffer_for_io(eb, bio_ctrl); - if (ret <= 0) { + if (!lock_extent_buffer_for_io(eb, bio_ctrl)) { btrfs_revert_meta_write_pointer(cache, eb); if (cache) btrfs_put_block_group(cache); From patchwork Thu Mar 30 06:30:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBFEBC74A5B for ; Thu, 30 Mar 2023 06:31:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230009AbjC3Gba (ORCPT ); Thu, 30 Mar 2023 02:31:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gb3 (ORCPT ); Thu, 30 Mar 2023 02:31:29 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8634B61AA for ; Wed, 29 Mar 2023 23:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=K+tR+e/pzfkKSNDm8HuiiKOMNlX5UWB+76wzv378VQM=; b=QnTEy2xiRXWjZbWmv6zL/3HL8u GC3r5rML8KTQ5NcSzaHI2F2v0Xg1VOKBEVgfhE+cb938LzFEJEAHPKrCjtRyACvQko2xYZOsNPK1A ljYcMnWWRwnHuBqTd7gwqk+C5XJVu6ivoHf7mOp3e7fv0/3mkmjlZ4UCKMjErF453DQ+ug7IznXMd XX6U3z7DLhGOfaf4DGlxJpwQIwkrBneLJgDsR0bynMvsdQJvjqwA4l9POeGujcLKlIBmgAHxEFudZ zXQEsAHfYbIsDi2LeQCKjSdNySk1ahHY9JK+tPmRyjnj0faHgy6j29W7P4S6xsfjXLpuhU0CbrHXw qvuzy+XQ==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlol-002lfC-08; Thu, 30 Mar 2023 06:31:27 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 11/21] btrfs: submit a writeback bio per extent_buffer Date: Thu, 30 Mar 2023 15:30:49 +0900 Message-Id: <20230330063059.1574380-12-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Stop trying to cluster writes of multiple extent_buffers into a single bio. There is no need for that as the blk_plug mechanism used all the way up in writeback_inodes_wb gives us the same I/O pattern even with multiple bios. Removing the clustering simplifies lock_extent_buffer_for_io a lot and will also allow passing the eb as private data to the end I/O handler. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 102 ++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 65 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 08e4e53f42e8a7..2d28744793c28d 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1626,41 +1626,24 @@ static void end_extent_buffer_writeback(struct extent_buffer *eb) /* * Lock extent buffer status and pages for writeback. * - * May try to flush write bio if we can't get the lock. - * * Return %false if the extent buffer doesn't need to be submitted (e.g. the * extent buffer is not dirty) * Return %true is the extent buffer is submitted to bio. */ static bool lock_extent_buffer_for_io(struct extent_buffer *eb, - struct btrfs_bio_ctrl *bio_ctrl) + struct writeback_control *wbc) { struct btrfs_fs_info *fs_info = eb->fs_info; - int i, num_pages; - int flush = 0; bool ret = false; + int i; - if (!btrfs_try_tree_write_lock(eb)) { - submit_write_bio(bio_ctrl, 0); - flush = 1; - btrfs_tree_lock(eb); - } - - if (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) { + btrfs_tree_lock(eb); + while (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) { btrfs_tree_unlock(eb); - if (bio_ctrl->wbc->sync_mode != WB_SYNC_ALL) + if (wbc->sync_mode != WB_SYNC_ALL) return false; - if (!flush) { - submit_write_bio(bio_ctrl, 0); - flush = 1; - } - while (1) { - wait_on_extent_buffer_writeback(eb); - btrfs_tree_lock(eb); - if (!test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) - break; - btrfs_tree_unlock(eb); - } + wait_on_extent_buffer_writeback(eb); + btrfs_tree_lock(eb); } /* @@ -1692,19 +1675,8 @@ static bool lock_extent_buffer_for_io(struct extent_buffer *eb, if (!ret || fs_info->nodesize < PAGE_SIZE) return ret; - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - struct page *p = eb->pages[i]; - - if (!trylock_page(p)) { - if (!flush) { - submit_write_bio(bio_ctrl, 0); - flush = 1; - } - lock_page(p); - } - } - + for (i = 0; i < num_extent_pages(eb); i++) + lock_page(eb->pages[i]); return ret; } @@ -1935,11 +1907,16 @@ static void prepare_eb_write(struct extent_buffer *eb) * Page locking is only utilized at minimum to keep the VMM code happy. */ static void write_one_subpage_eb(struct extent_buffer *eb, - struct btrfs_bio_ctrl *bio_ctrl) + struct writeback_control *wbc) { struct btrfs_fs_info *fs_info = eb->fs_info; struct page *page = eb->pages[0]; bool no_dirty_ebs = false; + struct btrfs_bio_ctrl bio_ctrl = { + .wbc = wbc, + .opf = REQ_OP_WRITE | wbc_to_write_flags(wbc), + .end_io_func = end_bio_subpage_eb_writepage, + }; prepare_eb_write(eb); @@ -1953,40 +1930,43 @@ static void write_one_subpage_eb(struct extent_buffer *eb, if (no_dirty_ebs) clear_page_dirty_for_io(page); - bio_ctrl->end_io_func = end_bio_subpage_eb_writepage; - - submit_extent_page(bio_ctrl, eb->start, page, eb->len, + submit_extent_page(&bio_ctrl, eb->start, page, eb->len, eb->start - page_offset(page)); unlock_page(page); + submit_one_bio(&bio_ctrl); /* * Submission finished without problem, if no range of the page is * dirty anymore, we have submitted a page. Update nr_written in wbc. */ if (no_dirty_ebs) - bio_ctrl->wbc->nr_to_write--; + wbc->nr_to_write--; } static noinline_for_stack void write_one_eb(struct extent_buffer *eb, - struct btrfs_bio_ctrl *bio_ctrl) + struct writeback_control *wbc) { u64 disk_bytenr = eb->start; int i, num_pages; + struct btrfs_bio_ctrl bio_ctrl = { + .wbc = wbc, + .opf = REQ_OP_WRITE | wbc_to_write_flags(wbc), + .end_io_func = end_bio_extent_buffer_writepage, + }; prepare_eb_write(eb); - bio_ctrl->end_io_func = end_bio_extent_buffer_writepage; - num_pages = num_extent_pages(eb); for (i = 0; i < num_pages; i++) { struct page *p = eb->pages[i]; clear_page_dirty_for_io(p); set_page_writeback(p); - submit_extent_page(bio_ctrl, disk_bytenr, p, PAGE_SIZE, 0); + submit_extent_page(&bio_ctrl, disk_bytenr, p, PAGE_SIZE, 0); disk_bytenr += PAGE_SIZE; - bio_ctrl->wbc->nr_to_write--; + wbc->nr_to_write--; unlock_page(p); } + submit_one_bio(&bio_ctrl); } /* @@ -2003,7 +1983,7 @@ static noinline_for_stack void write_one_eb(struct extent_buffer *eb, * Return >=0 for the number of submitted extent buffers. * Return <0 for fatal error. */ -static int submit_eb_subpage(struct page *page, struct btrfs_bio_ctrl *bio_ctrl) +static int submit_eb_subpage(struct page *page, struct writeback_control *wbc) { struct btrfs_fs_info *fs_info = btrfs_sb(page->mapping->host->i_sb); int submitted = 0; @@ -2055,8 +2035,8 @@ static int submit_eb_subpage(struct page *page, struct btrfs_bio_ctrl *bio_ctrl) if (!eb) continue; - if (lock_extent_buffer_for_io(eb, bio_ctrl)) { - write_one_subpage_eb(eb, bio_ctrl); + if (lock_extent_buffer_for_io(eb, wbc)) { + write_one_subpage_eb(eb, wbc); submitted++; } free_extent_buffer(eb); @@ -2084,7 +2064,7 @@ static int submit_eb_subpage(struct page *page, struct btrfs_bio_ctrl *bio_ctrl) * previous call. * Return <0 for fatal error. */ -static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, +static int submit_eb_page(struct page *page, struct writeback_control *wbc, struct extent_buffer **eb_context) { struct address_space *mapping = page->mapping; @@ -2096,7 +2076,7 @@ static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, return 0; if (btrfs_sb(page->mapping->host->i_sb)->nodesize < PAGE_SIZE) - return submit_eb_subpage(page, bio_ctrl); + return submit_eb_subpage(page, wbc); spin_lock(&mapping->private_lock); if (!PagePrivate(page)) { @@ -2129,8 +2109,7 @@ static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, * If for_sync, this hole will be filled with * trasnsaction commit. */ - if (bio_ctrl->wbc->sync_mode == WB_SYNC_ALL && - !bio_ctrl->wbc->for_sync) + if (wbc->sync_mode == WB_SYNC_ALL && !wbc->for_sync) ret = -EAGAIN; else ret = 0; @@ -2140,12 +2119,12 @@ static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, *eb_context = eb; - if (!lock_extent_buffer_for_io(eb, bio_ctrl)) { + if (!lock_extent_buffer_for_io(eb, wbc)) { btrfs_revert_meta_write_pointer(cache, eb); if (cache) btrfs_put_block_group(cache); free_extent_buffer(eb); - return ret; + return 0; } if (cache) { /* @@ -2154,7 +2133,7 @@ static int submit_eb_page(struct page *page, struct btrfs_bio_ctrl *bio_ctrl, btrfs_schedule_zone_finish_bg(cache, eb); btrfs_put_block_group(cache); } - write_one_eb(eb, bio_ctrl); + write_one_eb(eb, wbc); free_extent_buffer(eb); return 1; } @@ -2163,11 +2142,6 @@ int btree_write_cache_pages(struct address_space *mapping, struct writeback_control *wbc) { struct extent_buffer *eb_context = NULL; - struct btrfs_bio_ctrl bio_ctrl = { - .wbc = wbc, - .opf = REQ_OP_WRITE | wbc_to_write_flags(wbc), - .extent_locked = 0, - }; struct btrfs_fs_info *fs_info = BTRFS_I(mapping->host)->root->fs_info; int ret = 0; int done = 0; @@ -2209,7 +2183,7 @@ int btree_write_cache_pages(struct address_space *mapping, for (i = 0; i < nr_folios; i++) { struct folio *folio = fbatch.folios[i]; - ret = submit_eb_page(&folio->page, &bio_ctrl, &eb_context); + ret = submit_eb_page(&folio->page, wbc, &eb_context); if (ret == 0) continue; if (ret < 0) { @@ -2270,8 +2244,6 @@ int btree_write_cache_pages(struct address_space *mapping, ret = 0; if (!ret && BTRFS_FS_ERROR(fs_info)) ret = -EROFS; - submit_write_bio(&bio_ctrl, ret); - btrfs_zoned_meta_io_unlock(fs_info); return ret; } From patchwork Thu Mar 30 06:30:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7847C761AF for ; Thu, 30 Mar 2023 06:31:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229910AbjC3Gbc (ORCPT ); Thu, 30 Mar 2023 02:31:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbb (ORCPT ); Thu, 30 Mar 2023 02:31:31 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 647C161AA for ; Wed, 29 Mar 2023 23:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=n0jNulNBY7fKcGe0Rhb1Vn15MViVq18AjDg6zDFo3QI=; b=lZui8DAXL2nP7Q8ShoiVcQlgY3 eLtKF/PZMN4sXa+GAbK3iI/F1tEXgq9XzkSNS+CEHUE3B4jar1rnjoVI1KBMUw8pR4bbUron2j7yY 0eBqAk5QziaJtV+v3MTJJgHXiBjjsvFA7imMz3tL39JYOw8PHDhA84CG57e1Ru6wQoriDtfosAjfD ydYpqDxI7yji/D3740kn6bFFnWZpWvVG0J1+VvdNylBgxUzlGk2u0J+3Oc0l1rAyZjMb5p6ZigEiz nERYHeg6ny1R/sW6SCtVeW1F5i6x/ZpYuueK/2ANz1QZpqQcBNViihB5YhDMZ9KWNkKi8aZzkAQEW CbLDAD/g==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlon-002lfY-05; Thu, 30 Mar 2023 06:31:29 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 12/21] btrfs: move page locking from lock_extent_buffer_for_io to write_one_eb Date: Thu, 30 Mar 2023 15:30:50 +0900 Message-Id: <20230330063059.1574380-13-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Locking the pages in lock_extent_buffer_for_io only for the non-subpage case is very confusing. Move it to write_one_eb to mirror the subpage case and simplify the code. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 2d28744793c28d..56a2e6421b7189 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1635,7 +1635,6 @@ static bool lock_extent_buffer_for_io(struct extent_buffer *eb, { struct btrfs_fs_info *fs_info = eb->fs_info; bool ret = false; - int i; btrfs_tree_lock(eb); while (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)) { @@ -1663,20 +1662,7 @@ static bool lock_extent_buffer_for_io(struct extent_buffer *eb, } else { spin_unlock(&eb->refs_lock); } - btrfs_tree_unlock(eb); - - /* - * Either we don't need to submit any tree block, or we're submitting - * subpage eb. - * Subpage metadata doesn't use page locking at all, so we can skip - * the page locking. - */ - if (!ret || fs_info->nodesize < PAGE_SIZE) - return ret; - - for (i = 0; i < num_extent_pages(eb); i++) - lock_page(eb->pages[i]); return ret; } @@ -1959,6 +1945,7 @@ static noinline_for_stack void write_one_eb(struct extent_buffer *eb, for (i = 0; i < num_pages; i++) { struct page *p = eb->pages[i]; + lock_page(p); clear_page_dirty_for_io(p); set_page_writeback(p); submit_extent_page(&bio_ctrl, disk_bytenr, p, PAGE_SIZE, 0); From patchwork Thu Mar 30 06:30:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 467C9C6FD1D for ; Thu, 30 Mar 2023 06:31:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230063AbjC3Gbe (ORCPT ); Thu, 30 Mar 2023 02:31:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbc (ORCPT ); Thu, 30 Mar 2023 02:31:32 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 19BAC6585 for ; Wed, 29 Mar 2023 23:31:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Ruf1OekTMhobr3JG7I+L+ptQDTagB+T+iFbxdc9hBGM=; b=fFQPQgeRptHf7HWXuvEA8yTRJc NPjxAqZnxrofNjijrzDd64bQm2G3MVVPq32vHH6B1jQqe4CXDGUJBG4JosIxpPE4A42U+1dHihTh0 ng4xT7Qmqqv8MEvR0aUrV7T3XsHV3LGP0s10o1id2er15+WsMMhfjagoySDUgrdrAPGzU7vPyQSzO MUFfmIaZ0we9Cns/S7BUbBF2tLT7nwKozq+l6fYZQtpOeaGoAOh/E+LwA9CeEntRuzP071HCi27od mxt+7faF+R3omWTzhMJeNxPb9EfpMBsDrXyPbdtNJrvpv4jLggQyU9a9uQLb5B3009+Y/JfGYdW8/ aBq3tvIA==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloo-002lg2-2Q; Thu, 30 Mar 2023 06:31:30 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 13/21] btrfs: don't use btrfs_bio_ctrl for extent buffer writing Date: Thu, 30 Mar 2023 15:30:51 +0900 Message-Id: <20230330063059.1574380-14-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The btrfs_bio_ctrl machinery is overkill for writing extent_buffers as we always operate on PAGE SIZE chunks (or one smaller one for the subpage case) that are contigous and are guaranteed to fit into a single bio. Replace it with open coded btrfs_bio_alloc, __bio_add_page and btrfs_submit_bio calls. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/extent_io.c | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 56a2e6421b7189..f813ce5c7e14da 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -121,9 +121,6 @@ static void submit_one_bio(struct btrfs_bio_ctrl *bio_ctrl) /* Caller should ensure the bio has at least some range added */ ASSERT(bbio->bio.bi_iter.bi_size); - if (!is_data_inode(&bbio->inode->vfs_inode)) - bbio->bio.bi_opf |= REQ_META; - if (btrfs_op(&bbio->bio) == BTRFS_MAP_READ && bio_ctrl->compress_type != BTRFS_COMPRESS_NONE) btrfs_submit_compressed_read(bbio); @@ -1898,11 +1895,7 @@ static void write_one_subpage_eb(struct extent_buffer *eb, struct btrfs_fs_info *fs_info = eb->fs_info; struct page *page = eb->pages[0]; bool no_dirty_ebs = false; - struct btrfs_bio_ctrl bio_ctrl = { - .wbc = wbc, - .opf = REQ_OP_WRITE | wbc_to_write_flags(wbc), - .end_io_func = end_bio_subpage_eb_writepage, - }; + struct btrfs_bio *bbio; prepare_eb_write(eb); @@ -1916,10 +1909,16 @@ static void write_one_subpage_eb(struct extent_buffer *eb, if (no_dirty_ebs) clear_page_dirty_for_io(page); - submit_extent_page(&bio_ctrl, eb->start, page, eb->len, - eb->start - page_offset(page)); + bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, + REQ_OP_WRITE | REQ_META | wbc_to_write_flags(wbc), + BTRFS_I(eb->fs_info->btree_inode), + end_bio_subpage_eb_writepage, NULL); + bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; + bbio->file_offset = eb->start; + __bio_add_page(&bbio->bio, page, eb->len, eb->start - page_offset(page)); unlock_page(page); - submit_one_bio(&bio_ctrl); + btrfs_submit_bio(bbio, 0); + /* * Submission finished without problem, if no range of the page is * dirty anymore, we have submitted a page. Update nr_written in wbc. @@ -1931,16 +1930,18 @@ static void write_one_subpage_eb(struct extent_buffer *eb, static noinline_for_stack void write_one_eb(struct extent_buffer *eb, struct writeback_control *wbc) { - u64 disk_bytenr = eb->start; + struct btrfs_bio *bbio; int i, num_pages; - struct btrfs_bio_ctrl bio_ctrl = { - .wbc = wbc, - .opf = REQ_OP_WRITE | wbc_to_write_flags(wbc), - .end_io_func = end_bio_extent_buffer_writepage, - }; prepare_eb_write(eb); + bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, + REQ_OP_WRITE | REQ_META | wbc_to_write_flags(wbc), + BTRFS_I(eb->fs_info->btree_inode), + end_bio_extent_buffer_writepage, NULL); + bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; + bbio->file_offset = eb->start; + num_pages = num_extent_pages(eb); for (i = 0; i < num_pages; i++) { struct page *p = eb->pages[i]; @@ -1948,12 +1949,11 @@ static noinline_for_stack void write_one_eb(struct extent_buffer *eb, lock_page(p); clear_page_dirty_for_io(p); set_page_writeback(p); - submit_extent_page(&bio_ctrl, disk_bytenr, p, PAGE_SIZE, 0); - disk_bytenr += PAGE_SIZE; + __bio_add_page(&bbio->bio, p, PAGE_SIZE, 0); wbc->nr_to_write--; unlock_page(p); } - submit_one_bio(&bio_ctrl); + btrfs_submit_bio(bbio, 0); } /* From patchwork Thu Mar 30 06:30:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58349C6FD1D for ; Thu, 30 Mar 2023 06:31:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230260AbjC3Gbg (ORCPT ); Thu, 30 Mar 2023 02:31:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53960 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbf (ORCPT ); Thu, 30 Mar 2023 02:31:35 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3815F4EEA for ; Wed, 29 Mar 2023 23:31:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=25cRSKqOyX2nAcRD6Zx4sU93q3/LyKpc896Wg0bidcA=; b=Ef+NVP88ckopIDjVwnoY+sGSqE EgOHHhoaJqzOBoLdHVNLd5dfiPVHT71NBmoDJtUNzaMtpPxKfmd474dUu5NGmcoZoCra+KyRmCYku gxj57FvEwABvygTyb7DPmX0N3fKDix+u6h9zk5wnKGbMRBrPpI6i1l0P7w78rwE6t87HlrxZu7z5J m7oHDQeTd2V8EVtXUX+g8qc4tSgQYD2JAAwLcAFlr9sg+519AMPFX/cCKUo0VDm++I9KtN+adFEyg sS+IZlbFd0aS+V9Il4pRZ6uNagpG1DYovnXVDcaij2FvsbG5zasgkTqzQziLdiJknitcH/ugnQZbL SOtuuRDg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloq-002lgH-2B; Thu, 30 Mar 2023 06:31:33 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 14/21] btrfs: use a separate end_io handler for extent_buffer writing Date: Thu, 30 Mar 2023 15:30:52 +0900 Message-Id: <20230330063059.1574380-15-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Now that we always use a single bio to write an extent_buffer, the buffer can be passed to the end_io handler as private data. This allows to simplify the metadata write end I/O handler, and merge the subpage end_io handler into the main one. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 127 +++++++++---------------------------------- 1 file changed, 27 insertions(+), 100 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index f813ce5c7e14da..d306f3a2df146e 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1613,13 +1613,6 @@ void wait_on_extent_buffer_writeback(struct extent_buffer *eb) TASK_UNINTERRUPTIBLE); } -static void end_extent_buffer_writeback(struct extent_buffer *eb) -{ - clear_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags); - smp_mb__after_atomic(); - wake_up_bit(&eb->bflags, EXTENT_BUFFER_WRITEBACK); -} - /* * Lock extent buffer status and pages for writeback. * @@ -1663,13 +1656,11 @@ static bool lock_extent_buffer_for_io(struct extent_buffer *eb, return ret; } -static void set_btree_ioerr(struct page *page, struct extent_buffer *eb) +static void set_btree_ioerr(struct extent_buffer *eb) { struct btrfs_fs_info *fs_info = eb->fs_info; - btrfs_page_set_error(fs_info, page, eb->start, eb->len); - if (test_and_set_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags)) - return; + set_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags); /* * A read may stumble upon this buffer later, make sure that it gets an @@ -1683,7 +1674,7 @@ static void set_btree_ioerr(struct page *page, struct extent_buffer *eb) * return a 0 because we are readonly if we don't modify the err seq for * the superblock. */ - mapping_set_error(page->mapping, -EIO); + mapping_set_error(eb->fs_info->btree_inode->i_mapping, -EIO); /* * If writeback for a btree extent that doesn't belong to a log tree @@ -1758,102 +1749,38 @@ static struct extent_buffer *find_extent_buffer_nolock( return NULL; } -/* - * The endio function for subpage extent buffer write. - * - * Unlike end_bio_extent_buffer_writepage(), we only call end_page_writeback() - * after all extent buffers in the page has finished their writeback. - */ -static void end_bio_subpage_eb_writepage(struct btrfs_bio *bbio) +static void extent_buffer_write_end_io(struct btrfs_bio *bbio) { - struct bio *bio = &bbio->bio; - struct btrfs_fs_info *fs_info; - struct bio_vec *bvec; + struct extent_buffer *eb = bbio->private; + struct btrfs_fs_info *fs_info = eb->fs_info; + bool uptodate = !bbio->bio.bi_status; struct bvec_iter_all iter_all; + struct bio_vec *bvec; + u32 bio_offset = 0; - fs_info = btrfs_sb(bio_first_page_all(bio)->mapping->host->i_sb); - ASSERT(fs_info->nodesize < PAGE_SIZE); + if (!uptodate) + set_btree_ioerr(eb); - ASSERT(!bio_flagged(bio, BIO_CLONED)); - bio_for_each_segment_all(bvec, bio, iter_all) { + bio_for_each_segment_all(bvec, &bbio->bio, iter_all) { + u64 start = eb->start + bio_offset; struct page *page = bvec->bv_page; - u64 bvec_start = page_offset(page) + bvec->bv_offset; - u64 bvec_end = bvec_start + bvec->bv_len - 1; - u64 cur_bytenr = bvec_start; - - ASSERT(IS_ALIGNED(bvec->bv_len, fs_info->nodesize)); - - /* Iterate through all extent buffers in the range */ - while (cur_bytenr <= bvec_end) { - struct extent_buffer *eb; - int done; - - /* - * Here we can't use find_extent_buffer(), as it may - * try to lock eb->refs_lock, which is not safe in endio - * context. - */ - eb = find_extent_buffer_nolock(fs_info, cur_bytenr); - ASSERT(eb); - - cur_bytenr = eb->start + eb->len; - - ASSERT(test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags)); - done = atomic_dec_and_test(&eb->io_pages); - ASSERT(done); - - if (bio->bi_status || - test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags)) { - btrfs_page_clear_uptodate(fs_info, page, - eb->start, eb->len); - set_btree_ioerr(page, eb); - } + u32 len = bvec->bv_len; + + atomic_dec(&eb->io_pages); - btrfs_subpage_clear_writeback(fs_info, page, eb->start, - eb->len); - end_extent_buffer_writeback(eb); - /* - * free_extent_buffer() will grab spinlock which is not - * safe in endio context. Thus here we manually dec - * the ref. - */ - atomic_dec(&eb->refs); + if (!uptodate) { + btrfs_page_clear_uptodate(fs_info, page, start, len); + btrfs_page_set_error(fs_info, page, start, len); } + btrfs_page_clear_writeback(fs_info, page, start, len); + bio_offset += len; } - bio_put(bio); -} - -static void end_bio_extent_buffer_writepage(struct btrfs_bio *bbio) -{ - struct bio *bio = &bbio->bio; - struct bio_vec *bvec; - struct extent_buffer *eb; - int done; - struct bvec_iter_all iter_all; - ASSERT(!bio_flagged(bio, BIO_CLONED)); - bio_for_each_segment_all(bvec, bio, iter_all) { - struct page *page = bvec->bv_page; - - eb = (struct extent_buffer *)page->private; - BUG_ON(!eb); - done = atomic_dec_and_test(&eb->io_pages); - - if (bio->bi_status || - test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags)) { - ClearPageUptodate(page); - set_btree_ioerr(page, eb); - } - - end_page_writeback(page); - - if (!done) - continue; - - end_extent_buffer_writeback(eb); - } + clear_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags); + smp_mb__after_atomic(); + wake_up_bit(&eb->bflags, EXTENT_BUFFER_WRITEBACK); - bio_put(bio); + bio_put(&bbio->bio); } static void prepare_eb_write(struct extent_buffer *eb) @@ -1912,7 +1839,7 @@ static void write_one_subpage_eb(struct extent_buffer *eb, bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, REQ_OP_WRITE | REQ_META | wbc_to_write_flags(wbc), BTRFS_I(eb->fs_info->btree_inode), - end_bio_subpage_eb_writepage, NULL); + extent_buffer_write_end_io, eb); bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; bbio->file_offset = eb->start; __bio_add_page(&bbio->bio, page, eb->len, eb->start - page_offset(page)); @@ -1938,7 +1865,7 @@ static noinline_for_stack void write_one_eb(struct extent_buffer *eb, bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, REQ_OP_WRITE | REQ_META | wbc_to_write_flags(wbc), BTRFS_I(eb->fs_info->btree_inode), - end_bio_extent_buffer_writepage, NULL); + extent_buffer_write_end_io, eb); bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; bbio->file_offset = eb->start; From patchwork Thu Mar 30 06:30:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB8CFC761AF for ; Thu, 30 Mar 2023 06:31:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230264AbjC3Gbi (ORCPT ); Thu, 30 Mar 2023 02:31:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbg (ORCPT ); Thu, 30 Mar 2023 02:31:36 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E61A261A4 for ; Wed, 29 Mar 2023 23:31:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=GTXPl7ZnHJt9YGS+zBSwtJqc2OTNzbrNHVzIoAO8mfY=; b=GL4RV2BjWa0h5Q9YvqiX8hHYxE SxevWTzSw4Cl9PQGFGVNy0SkomMQL/0V+XEk7WbPkA3dnBwEFLAxNpIioIvnq07vCpdHCyn/NJ1q6 btUr080DRhexh5jYnNLUosuLGpb3R8QAZmIJE7+lGSPxtveOHzA3q+uO29+u/S54k42xOeuFITjVO sxDxOAwNGSMPWKCTLd6bZLQ11o4kWXeJnixFPQwfweH9oE8L5FCI4M/0LHMJCR5+4JXUPKHeyAAqo jOmv8tBNlr+M1FzviZysR1r6DjukQPikzBpu1vgqmB7Em1eG57axcLR6o/WITIOYcCPbhdDNoNtAM 17sRp9WA==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlos-002lgj-1n; Thu, 30 Mar 2023 06:31:34 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn , Qu Wenruo Subject: [PATCH 15/21] btrfs: remove the extent_buffer lookup in btree block checksumming Date: Thu, 30 Mar 2023 15:30:53 +0900 Message-Id: <20230330063059.1574380-16-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The checksumming of btree blocks always operates on the entire extent_buffer, and because btree blocks are always allocated contiguously on disk they are never split by btrfs_submit_bio. Simplify the checksumming code by finding the extent_buffer in the btrfs_bio private data instead of trying to search through the bio_vec. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn Reviewed-by: Qu Wenruo --- fs/btrfs/disk-io.c | 121 ++++++++++----------------------------------- 1 file changed, 25 insertions(+), 96 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 4fd181f67925f9..3e15965c531e81 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -312,12 +312,35 @@ int btrfs_read_extent_buffer(struct extent_buffer *eb, return ret; } -static int csum_one_extent_buffer(struct extent_buffer *eb) +/* + * Checksum a dirty tree block before IO. + */ +blk_status_t btree_csum_one_bio(struct btrfs_bio *bbio) { + struct extent_buffer *eb = bbio->private; struct btrfs_fs_info *fs_info = eb->fs_info; + u64 found_start = btrfs_header_bytenr(eb); u8 result[BTRFS_CSUM_SIZE]; int ret; + /* + * Btree blocks are always contiguous on disk. + */ + if (WARN_ON_ONCE(bbio->file_offset != eb->start)) + return BLK_STS_IOERR; + if (WARN_ON_ONCE(bbio->bio.bi_iter.bi_size != eb->len)) + return BLK_STS_IOERR; + + if (test_bit(EXTENT_BUFFER_NO_CHECK, &eb->bflags)) { + WARN_ON_ONCE(found_start != 0); + return BLK_STS_OK; + } + + if (WARN_ON_ONCE(found_start != eb->start)) + return BLK_STS_IOERR; + if (WARN_ON_ONCE(!PageUptodate(eb->pages[0]))) + return BLK_STS_IOERR; + ASSERT(memcmp_extent_buffer(eb, fs_info->fs_devices->metadata_uuid, offsetof(struct btrfs_header, fsid), BTRFS_FSID_SIZE) == 0); @@ -344,8 +367,7 @@ static int csum_one_extent_buffer(struct extent_buffer *eb) goto error; } write_extent_buffer(eb, result, 0, fs_info->csum_size); - - return 0; + return BLK_STS_OK; error: btrfs_print_tree(eb, 0); @@ -359,99 +381,6 @@ static int csum_one_extent_buffer(struct extent_buffer *eb) */ WARN_ON(IS_ENABLED(CONFIG_BTRFS_DEBUG) || btrfs_header_owner(eb) == BTRFS_TREE_LOG_OBJECTID); - return ret; -} - -/* Checksum all dirty extent buffers in one bio_vec */ -static int csum_dirty_subpage_buffers(struct btrfs_fs_info *fs_info, - struct bio_vec *bvec) -{ - struct page *page = bvec->bv_page; - u64 bvec_start = page_offset(page) + bvec->bv_offset; - u64 cur; - int ret = 0; - - for (cur = bvec_start; cur < bvec_start + bvec->bv_len; - cur += fs_info->nodesize) { - struct extent_buffer *eb; - bool uptodate; - - eb = find_extent_buffer(fs_info, cur); - uptodate = btrfs_subpage_test_uptodate(fs_info, page, cur, - fs_info->nodesize); - - /* A dirty eb shouldn't disappear from buffer_radix */ - if (WARN_ON(!eb)) - return -EUCLEAN; - - if (WARN_ON(cur != btrfs_header_bytenr(eb))) { - free_extent_buffer(eb); - return -EUCLEAN; - } - if (WARN_ON(!uptodate)) { - free_extent_buffer(eb); - return -EUCLEAN; - } - - ret = csum_one_extent_buffer(eb); - free_extent_buffer(eb); - if (ret < 0) - return ret; - } - return ret; -} - -/* - * Checksum a dirty tree block before IO. This has extra checks to make sure - * we only fill in the checksum field in the first page of a multi-page block. - * For subpage extent buffers we need bvec to also read the offset in the page. - */ -static int csum_dirty_buffer(struct btrfs_fs_info *fs_info, struct bio_vec *bvec) -{ - struct page *page = bvec->bv_page; - u64 start = page_offset(page); - u64 found_start; - struct extent_buffer *eb; - - if (fs_info->nodesize < PAGE_SIZE) - return csum_dirty_subpage_buffers(fs_info, bvec); - - eb = (struct extent_buffer *)page->private; - if (page != eb->pages[0]) - return 0; - - found_start = btrfs_header_bytenr(eb); - - if (test_bit(EXTENT_BUFFER_NO_CHECK, &eb->bflags)) { - WARN_ON(found_start != 0); - return 0; - } - - /* - * Please do not consolidate these warnings into a single if. - * It is useful to know what went wrong. - */ - if (WARN_ON(found_start != start)) - return -EUCLEAN; - if (WARN_ON(!PageUptodate(page))) - return -EUCLEAN; - - return csum_one_extent_buffer(eb); -} - -blk_status_t btree_csum_one_bio(struct btrfs_bio *bbio) -{ - struct btrfs_fs_info *fs_info = bbio->inode->root->fs_info; - struct bvec_iter iter; - struct bio_vec bv; - int ret = 0; - - bio_for_each_segment(bv, &bbio->bio, iter) { - ret = csum_dirty_buffer(fs_info, &bv); - if (ret) - break; - } - return errno_to_blk_status(ret); } From patchwork Thu Mar 30 06:30:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6B42C74A5B for ; Thu, 30 Mar 2023 06:31:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230268AbjC3Gbl (ORCPT ); Thu, 30 Mar 2023 02:31:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54034 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbj (ORCPT ); Thu, 30 Mar 2023 02:31:39 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C5C654ED2 for ; Wed, 29 Mar 2023 23:31:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=uRZfLCor618ENoYKE1v2Xpi8SudINs70durgTA+fWZ4=; b=jdDXLm0aYeJRXikJoCKT4TPkgr fyTpEqXwE3zemn8jXu2tpODb35uYWs4X/IZhmyovUpCQKsxr/nuj+MvVUQcWWo+F5XNv+yEKxl0Sc NZNaK1SHN3xO62FsYsYpuj3/r6o8dH6foWj5ZThVOuZJdRtBvNxZ4bPzRQfwZgfUsiaAoIT9+cCKb LqpofMj28UElxNq0sMM7FyweH/U2SiMQ68fSk0pbHn61Pq/m7SxPwsDzn+/I5EoW3gyPwXJ0htEgV MbTEGTwN+yUn8MD5sLR8+zDyBipiTWAjPJAJephAfDtfJmkQBuDsO6X+UVvTWlnqzFE40xg6JR/GJ cb32/OUg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlou-002lhF-17; Thu, 30 Mar 2023 06:31:36 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 16/21] btrfs: remove the io_pages field in struct extent_buffer Date: Thu, 30 Mar 2023 15:30:54 +0900 Message-Id: <20230330063059.1574380-17-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org No need to track the number of pages under I/O now that each extent_buffer is read and written using a single bio. For the read side we need to grab an extra reference for the duration of the I/O to prevent eviction, though. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 17 +++++------------ fs/btrfs/extent_io.h | 1 - 2 files changed, 5 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index d306f3a2df146e..920630bf7af82b 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1766,8 +1766,6 @@ static void extent_buffer_write_end_io(struct btrfs_bio *bbio) struct page *page = bvec->bv_page; u32 len = bvec->bv_len; - atomic_dec(&eb->io_pages); - if (!uptodate) { btrfs_page_clear_uptodate(fs_info, page, start, len); btrfs_page_set_error(fs_info, page, start, len); @@ -1790,7 +1788,6 @@ static void prepare_eb_write(struct extent_buffer *eb) unsigned long end; clear_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags); - atomic_set(&eb->io_pages, num_extent_pages(eb)); /* Set btree blocks beyond nritems with 0 to avoid stale content */ nritems = btrfs_header_nritems(eb); @@ -3230,8 +3227,7 @@ static void __free_extent_buffer(struct extent_buffer *eb) static int extent_buffer_under_io(const struct extent_buffer *eb) { - return (atomic_read(&eb->io_pages) || - test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) || + return (test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) || test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)); } @@ -3368,7 +3364,6 @@ __alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start, spin_lock_init(&eb->refs_lock); atomic_set(&eb->refs, 1); - atomic_set(&eb->io_pages, 0); ASSERT(len <= BTRFS_MAX_METADATA_BLOCKSIZE); @@ -3485,9 +3480,9 @@ static void check_buffer_tree_ref(struct extent_buffer *eb) * adequately protected by the refcount, but the TREE_REF bit and * its corresponding reference are not. To protect against this * class of races, we call check_buffer_tree_ref from the codepaths - * which trigger io after they set eb->io_pages. Note that once io is - * initiated, TREE_REF can no longer be cleared, so that is the - * moment at which any such race is best fixed. + * which trigger io. Note that once io is initiated, TREE_REF can no + * longer be cleared, so that is the moment at which any such race is + * best fixed. */ refs = atomic_read(&eb->refs); if (refs >= 2 && test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) @@ -4057,7 +4052,6 @@ static void extent_buffer_read_end_io(struct btrfs_bio *bbio) struct bio_vec *bvec; u32 bio_offset = 0; - atomic_inc(&eb->refs); eb->read_mirror = bbio->mirror_num; if (uptodate && @@ -4072,7 +4066,6 @@ static void extent_buffer_read_end_io(struct btrfs_bio *bbio) } bio_for_each_segment_all(bvec, &bbio->bio, iter_all) { - atomic_dec(&eb->io_pages); end_page_read(bvec->bv_page, uptodate, eb->start + bio_offset, bvec->bv_len); bio_offset += bvec->bv_len; @@ -4095,8 +4088,8 @@ static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); eb->read_mirror = 0; - atomic_set(&eb->io_pages, num_pages); check_buffer_tree_ref(eb); + atomic_inc(&eb->refs); bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, REQ_OP_READ | REQ_META, diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 342412d37a7b4b..12854a2b48f060 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -79,7 +79,6 @@ struct extent_buffer { struct btrfs_fs_info *fs_info; spinlock_t refs_lock; atomic_t refs; - atomic_t io_pages; int read_mirror; struct rcu_head rcu_head; pid_t lock_owner; From patchwork Thu Mar 30 06:30:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A4AEC6FD1D for ; Thu, 30 Mar 2023 06:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230064AbjC3Gbo (ORCPT ); Thu, 30 Mar 2023 02:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230270AbjC3Gbm (ORCPT ); Thu, 30 Mar 2023 02:31:42 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B97C96A54 for ; Wed, 29 Mar 2023 23:31:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=o+W3XKxYHk4PweP/VrO9/pxPbUP7eOBZZoWg4aMZgp4=; b=RLQPkB9aBmJ99d8FFlZLQ3wYEH BheXeFgJYY4IBdrrVSnMKqY8JgN0caEoj8xCpY022w+8vm0CZTkp+0f/uzQIqy05btEMnLV06munj dGBNHNgrr3enID74EWTvAVmBTn5YcnSW6+752Pm3+z8FGAz2OuTpZGSxo1vnSdBg16Ngo92fXeajh A8FtPrDLTvVrbXoa1Kb51LU0zP1F7zGXPHIrnxaf3LaMpaFNkV1XpMwnI4OxdYFwX+zowQ3TYybt3 2cdFcAiVuqFKulT2AEU/+cuE+xZVI4v2loUEhnKwZ6KeyLYSjpLt5JaWT1Yz2c4HVC5EEUJrJI+0q QEGcH+Mg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlow-002lhl-21; Thu, 30 Mar 2023 06:31:38 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 17/21] btrfs: stop using PageError for extent_buffers Date: Thu, 30 Mar 2023 15:30:55 +0900 Message-Id: <20230330063059.1574380-18-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org PageError is only used to limit the uptodate check in assert_eb_page_uptodate. But we have a much more useful flag indicating the exact condition we are about with the EXTENT_BUFFER_WRITE_ERR flag, so use that instead and help the kernel torward eventually removing PageError. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 25 +++++++++---------------- 1 file changed, 9 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 920630bf7af82b..642e954ac99259 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1766,10 +1766,8 @@ static void extent_buffer_write_end_io(struct btrfs_bio *bbio) struct page *page = bvec->bv_page; u32 len = bvec->bv_len; - if (!uptodate) { + if (!uptodate) btrfs_page_clear_uptodate(fs_info, page, start, len); - btrfs_page_set_error(fs_info, page, start, len); - } btrfs_page_clear_writeback(fs_info, page, start, len); bio_offset += len; } @@ -4102,10 +4100,8 @@ static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, __bio_add_page(&bbio->bio, eb->pages[0], eb->len, eb->start - page_offset(eb->pages[0])); } else { - for (i = 0; i < num_pages; i++) { - ClearPageError(eb->pages[i]); + for (i = 0; i < num_pages; i++) __bio_add_page(&bbio->bio, eb->pages[i], PAGE_SIZE, 0); - } } btrfs_submit_bio(bbio, mirror_num); } @@ -4145,7 +4141,6 @@ static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, return 0; } - btrfs_subpage_clear_error(fs_info, page, eb->start, eb->len); btrfs_subpage_start_reader(fs_info, page, eb->start, eb->len); __read_extent_buffer_pages(eb, mirror_num, check); @@ -4387,18 +4382,16 @@ static void assert_eb_page_uptodate(const struct extent_buffer *eb, * looked up. We don't want to complain in this case, as the page was * valid before, we just didn't write it out. Instead we want to catch * the case where we didn't actually read the block properly, which - * would have !PageUptodate && !PageError, as we clear PageError before - * reading. + * would have !PageUptodate && !EXTENT_BUFFER_WRITE_ERR. */ - if (fs_info->nodesize < PAGE_SIZE) { - bool uptodate, error; + if (test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags)) + return; - uptodate = btrfs_subpage_test_uptodate(fs_info, page, - eb->start, eb->len); - error = btrfs_subpage_test_error(fs_info, page, eb->start, eb->len); - WARN_ON(!uptodate && !error); + if (fs_info->nodesize < PAGE_SIZE) { + WARN_ON(!btrfs_subpage_test_uptodate(fs_info, page, + eb->start, eb->len)); } else { - WARN_ON(!PageUptodate(page) && !PageError(page)); + WARN_ON(!PageUptodate(page)); } } From patchwork Thu Mar 30 06:30:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FC6C74A5B for ; Thu, 30 Mar 2023 06:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230270AbjC3Gbo (ORCPT ); Thu, 30 Mar 2023 02:31:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbn (ORCPT ); Thu, 30 Mar 2023 02:31:43 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1B2665240 for ; Wed, 29 Mar 2023 23:31:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=ix9SazV3GqXRW4nSmg6a1NKhf+/6VN2YsGmTiTD5fuE=; b=iTpX89HiRF5mJ06NhKVyUpj+qQ xiHS/l0YWfEDz5tvMrpztsEuo8uSOWJuY+wieAj8uoXcIbdFDDju0XcseaFgiRXPWauii7u4RLH0S 6EIWAv/6jqtgvy7HX6okRMq/dE3vGUObEorW7xobjoLHhdG4cRXiIWYdX6tC/XHZU5o/lvrPRcWvR bXQgLpPQDdZPrYfp77D84SvepY9MbRcWJ4Pe3MkXinka/EB29EZoWqGb6x9YduYx5DTYosrlNSyJt s7UZgexRsJKCvPu+I2L1e8xZM1F1Dg9EBN6nLwqlu3DhqSSZwYandgYGFO/1yWvf2Ug94byAecXrD SZgl0uhg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phloy-002liE-38; Thu, 30 Mar 2023 06:31:41 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 18/21] btrfs: don't check for uptodate pages in read_extent_buffer_pages Date: Thu, 30 Mar 2023 15:30:56 +0900 Message-Id: <20230330063059.1574380-19-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The only place that reads in pages and thus marks them uptodate for the btree inode is read_extent_buffer_pages. Which means that either pages are already uptodate from an old buffer when creating a new one in alloc_extent_buffer, or they will be updated by ca call to read_extent_buffer_pages. This means the checks for uptodate pages in read_extent_buffer_pages and read_extent_buffer_subpage are superfluous and can be removed. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 21 +-------------------- 1 file changed, 1 insertion(+), 20 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 642e954ac99259..587ae974c39dd9 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4132,10 +4132,7 @@ static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, return ret; } - if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags) || - PageUptodate(page) || - btrfs_subpage_test_uptodate(fs_info, page, eb->start, eb->len)) { - set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); + if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) { unlock_extent(io_tree, eb->start, eb->start + eb->len - 1, &cached_state); return 0; @@ -4162,7 +4159,6 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, int i; struct page *page; int locked_pages = 0; - int all_uptodate = 1; int num_pages; if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) @@ -4197,21 +4193,6 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, } locked_pages++; } - /* - * We need to firstly lock all pages to make sure that - * the uptodate bit of our pages won't be affected by - * clear_extent_buffer_uptodate(). - */ - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - if (!PageUptodate(page)) - all_uptodate = 0; - } - - if (all_uptodate) { - set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); - goto unlock_exit; - } __read_extent_buffer_pages(eb, mirror_num, check); From patchwork Thu Mar 30 06:30:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03D6BC6FD1D for ; Thu, 30 Mar 2023 06:31:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229773AbjC3Gbr (ORCPT ); Thu, 30 Mar 2023 02:31:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229517AbjC3Gbq (ORCPT ); Thu, 30 Mar 2023 02:31:46 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 28E3061AA for ; Wed, 29 Mar 2023 23:31:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=mH1uJZL/fH9NEAFk7hzMyXMWOiKvEdF6TfnpUfLqRzk=; b=ASW+aVUVP3cbNI9afRVngQcmpV GJeN/59Vbb4DkZoC+vAR9W7laB3tJFLr10T+Y5BKDLt5SpbyyMTZ523XNDwCERuq3O45ieVGOAFnk f7NBp6Y4pAs2E7YMWS1EvLCXIGIXqrl+BeYmFPbEyNDS7WBfoy8shbXdYW/JP46GIHdMvybUkbLV0 fLQ7x+R9tY2wjpzUjRW8HmZ5GhFjqGEmxQcZNkqC9uJHNvRzQ516t3aSYofyr9e+3HNI01bBq1tqm 23zy4N2ecqPcgdVEdZ+F+6ONe9iVmr64uwAwaE28SiCRgeDk6FaiXW0I5lCOzIqPeP/Tz5qg8YqG8 T5J7ejyA==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlp1-002lie-0y; Thu, 30 Mar 2023 06:31:43 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org Subject: [PATCH 19/21] btrfs: stop using lock_extent in btrfs_buffer_uptodate Date: Thu, 30 Mar 2023 15:30:57 +0900 Message-Id: <20230330063059.1574380-20-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org The only other place that locks extents on the btree inode is read_extent_buffer_subpage while reading in the partial page for a buffer. This means locking the extent in btrfs_buffer_uptodate does not synchronize with anything on non-supage file systems, and on subpage file systems it only waits for a parallel read(-ahead) to finish, which seems to be counter to what the callers actually expect. Signed-off-by: Christoph Hellwig --- fs/btrfs/disk-io.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 3e15965c531e81..3bce00803af75f 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -113,11 +113,6 @@ static void csum_tree_block(struct extent_buffer *buf, u8 *result) int btrfs_buffer_uptodate(struct extent_buffer *eb, u64 parent_transid, int atomic) { - struct inode *btree_inode = eb->pages[0]->mapping->host; - struct extent_io_tree *io_tree = &BTRFS_I(btree_inode)->io_tree; - struct extent_state *cached_state = NULL; - int ret = 1; - if (!extent_buffer_uptodate(eb)) return 0; @@ -127,7 +122,6 @@ int btrfs_buffer_uptodate(struct extent_buffer *eb, u64 parent_transid, if (atomic) return -EAGAIN; - lock_extent(io_tree, eb->start, eb->start + eb->len - 1, &cached_state); if (!extent_buffer_uptodate(eb) || btrfs_header_generation(eb) != parent_transid) { btrfs_err_rl(eb->fs_info, @@ -135,11 +129,9 @@ int btrfs_buffer_uptodate(struct extent_buffer *eb, u64 parent_transid, eb->start, eb->read_mirror, parent_transid, btrfs_header_generation(eb)); clear_extent_buffer_uptodate(eb); - ret = 0; + return 0; } - unlock_extent(io_tree, eb->start, eb->start + eb->len - 1, - &cached_state); - return ret; + return 1; } static bool btrfs_supported_super_csum(u16 csum_type) From patchwork Thu Mar 30 06:30:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EE71C74A5B for ; Thu, 30 Mar 2023 06:31:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229875AbjC3Gbs (ORCPT ); Thu, 30 Mar 2023 02:31:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54206 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230272AbjC3Gbr (ORCPT ); Thu, 30 Mar 2023 02:31:47 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 302AF61A4 for ; Wed, 29 Mar 2023 23:31:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Y2cDbySJk4MGj3ybsV1lqp9RNM+GQVj3tLfk973dazk=; b=knr2Wbjxi3dw+ciCNtIJJeIya1 i5ko/JcJNGN2EIZBaJAlHiT86MmZkKeVmkM5lQiSBRduEMB7u9fynY6xzWgKr80gchDuiOGlI18Ts Um4yhUTCDbecLIZNb4YQ4aTNkAGYGjwEHMy+pXPHri3yMqoKlZgQk7XCpAFwp7eV9aV84qR0ufUYI PAxja+z0XZABNifcLIemR4Um+dXCGGQN/6cFeKPBM0r082/OkAtvcJcYN3zGUPtGO9t+AB7kLeka4 rrTLnc9Lx2mDhP4arH8GdNqJ47oF0A7AjFASAkCaR5Xd5ZwIFzRz5UvjoU19lMhjhcdgdVmbFXP3X CxAmA1CQ==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlp2-002ljA-2n; Thu, 30 Mar 2023 06:31:45 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org Subject: [PATCH 20/21] btrfs: use per-buffer locking for extent_buffer reading Date: Thu, 30 Mar 2023 15:30:58 +0900 Message-Id: <20230330063059.1574380-21-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Instead of locking and unlocking every page or the extent, just add a new EXTENT_BUFFER_READING bit that mirrors EXTENT_BUFFER_WRITEBACK for synchronizing threads trying to read an extent_buffer and to wait for I/O completion. Signed-off-by: Christoph Hellwig --- fs/btrfs/extent_io.c | 153 +++++++++++-------------------------------- fs/btrfs/extent_io.h | 1 + 2 files changed, 39 insertions(+), 115 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 587ae974c39dd9..da4ea02a56c8bf 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -4045,6 +4045,7 @@ void set_extent_buffer_uptodate(struct extent_buffer *eb) static void extent_buffer_read_end_io(struct btrfs_bio *bbio) { struct extent_buffer *eb = bbio->private; + struct btrfs_fs_info *fs_info = eb->fs_info; bool uptodate = !bbio->bio.bi_status; struct bvec_iter_all iter_all; struct bio_vec *bvec; @@ -4064,26 +4065,49 @@ static void extent_buffer_read_end_io(struct btrfs_bio *bbio) } bio_for_each_segment_all(bvec, &bbio->bio, iter_all) { - end_page_read(bvec->bv_page, uptodate, eb->start + bio_offset, - bvec->bv_len); - bio_offset += bvec->bv_len; - } + u64 start = eb->start + bio_offset; + struct page *page = bvec->bv_page; + u32 len = bvec->bv_len; - if (eb->fs_info->nodesize < PAGE_SIZE) { - unlock_extent(&bbio->inode->io_tree, eb->start, - eb->start + bio_offset - 1, NULL); + if (uptodate) + btrfs_page_set_uptodate(fs_info, page, start, len); + else + btrfs_page_clear_uptodate(fs_info, page, start, len); + + bio_offset += len; } + + clear_bit(EXTENT_BUFFER_READING, &eb->bflags); + smp_mb__after_atomic(); + wake_up_bit(&eb->bflags, EXTENT_BUFFER_READING); free_extent_buffer(eb); bio_put(&bbio->bio); } -static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, - struct btrfs_tree_parent_check *check) +int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, + struct btrfs_tree_parent_check *check) { int num_pages = num_extent_pages(eb), i; struct btrfs_bio *bbio; + if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) + return 0; + + /* + * We could have had EXTENT_BUFFER_UPTODATE cleared by the write + * operation, which could potentially still be in flight. In this case + * we simply want to return an error. + */ + if (unlikely(test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags))) + return -EIO; + + /* + * Someone else is already reading the buffer, just wait for it. + */ + if (test_and_set_bit(EXTENT_BUFFER_READING, &eb->bflags)) + goto done; + clear_bit(EXTENT_BUFFER_READ_ERR, &eb->bflags); eb->read_mirror = 0; check_buffer_tree_ref(eb); @@ -4104,117 +4128,16 @@ static void __read_extent_buffer_pages(struct extent_buffer *eb, int mirror_num, __bio_add_page(&bbio->bio, eb->pages[i], PAGE_SIZE, 0); } btrfs_submit_bio(bbio, mirror_num); -} - -static int read_extent_buffer_subpage(struct extent_buffer *eb, int wait, - int mirror_num, - struct btrfs_tree_parent_check *check) -{ - struct btrfs_fs_info *fs_info = eb->fs_info; - struct extent_io_tree *io_tree; - struct page *page = eb->pages[0]; - struct extent_state *cached_state = NULL; - int ret; - - ASSERT(!test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags)); - ASSERT(PagePrivate(page)); - ASSERT(check); - io_tree = &BTRFS_I(fs_info->btree_inode)->io_tree; - - if (wait == WAIT_NONE) { - if (!try_lock_extent(io_tree, eb->start, eb->start + eb->len - 1, - &cached_state)) - return -EAGAIN; - } else { - ret = lock_extent(io_tree, eb->start, eb->start + eb->len - 1, - &cached_state); - if (ret < 0) - return ret; - } - - if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) { - unlock_extent(io_tree, eb->start, eb->start + eb->len - 1, - &cached_state); - return 0; - } - btrfs_subpage_start_reader(fs_info, page, eb->start, eb->len); - - __read_extent_buffer_pages(eb, mirror_num, check); - if (wait != WAIT_COMPLETE) { - free_extent_state(cached_state); - return 0; - } - - wait_extent_bit(io_tree, eb->start, eb->start + eb->len - 1, - EXTENT_LOCKED, &cached_state); - if (!test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) - return -EIO; - return 0; -} - -int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num, - struct btrfs_tree_parent_check *check) -{ - int i; - struct page *page; - int locked_pages = 0; - int num_pages; - - if (test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) - return 0; - - /* - * We could have had EXTENT_BUFFER_UPTODATE cleared by the write - * operation, which could potentially still be in flight. In this case - * we simply want to return an error. - */ - if (unlikely(test_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags))) - return -EIO; - - if (eb->fs_info->nodesize < PAGE_SIZE) - return read_extent_buffer_subpage(eb, wait, mirror_num, check); - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - if (wait == WAIT_NONE) { - /* - * WAIT_NONE is only utilized by readahead. If we can't - * acquire the lock atomically it means either the eb - * is being read out or under modification. - * Either way the eb will be or has been cached, - * readahead can exit safely. - */ - if (!trylock_page(page)) - goto unlock_exit; - } else { - lock_page(page); - } - locked_pages++; - } - - __read_extent_buffer_pages(eb, mirror_num, check); - - if (wait != WAIT_COMPLETE) - return 0; - - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - wait_on_page_locked(page); - if (!PageUptodate(page)) +done: + if (wait == WAIT_COMPLETE) { + wait_on_bit_io(&eb->bflags, EXTENT_BUFFER_READING, + TASK_UNINTERRUPTIBLE); + if (!test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags)) return -EIO; } return 0; - -unlock_exit: - while (locked_pages > 0) { - locked_pages--; - page = eb->pages[locked_pages]; - unlock_page(page); - } - return 0; } static bool report_eb_range(const struct extent_buffer *eb, unsigned long start, diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 12854a2b48f060..44f63ab18b1888 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -29,6 +29,7 @@ enum { /* write IO error */ EXTENT_BUFFER_WRITE_ERR, EXTENT_BUFFER_NO_CHECK, + EXTENT_BUFFER_READING, }; /* these are flags for __process_pages_contig */ From patchwork Thu Mar 30 06:30:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoph Hellwig X-Patchwork-Id: 13193455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95EABC761AF for ; Thu, 30 Mar 2023 06:31:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230280AbjC3Gbu (ORCPT ); Thu, 30 Mar 2023 02:31:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54248 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230018AbjC3Gbt (ORCPT ); Thu, 30 Mar 2023 02:31:49 -0400 Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80CAF4EFC for ; Wed, 29 Mar 2023 23:31:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=RgQmaP0oUs/FGyvrH4U0lCSwOBJyz0/By2BUIojaP/A=; b=aMQL3dK306LfeDKFJEX4YFOa/q 8PMnPu9c1PSxpFOSOO1QYJbCe/c7Gw7h0UJZVqjrbqbvUpn1F94PcynxJa3QKWGeckBEwWFqlDqhZ Jbzsnex1kZrvCh5cha1P3W2D/GrV9xz5WHuUhSw0zRu0xglFYqfE3Txr4N4jRDs0yPx0efbkqj5BG d6rh/CXEJFE7SvIDEbTseDVJA3TN2Y5aGlGYEieZKh9Pp2G/5l/8RXE/hBv83DuJbnx2K6JUD0AQC wmUl1EF1rH2EPxcFacfWx0BqPuJzallYWzd8yzsCL47cMx+cbUrcV6H/hYUbRjslg5mLd8AZ+noNR kXC7bNdg==; Received: from [182.171.77.115] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1phlp5-002ljn-0P; Thu, 30 Mar 2023 06:31:47 +0000 From: Christoph Hellwig To: Chris Mason , Josef Bacik , David Sterba Cc: linux-btrfs@vger.kernel.org, Johannes Thumshirn Subject: [PATCH 21/21] btrfs: merge write_one_subpage_eb into write_one_eb Date: Thu, 30 Mar 2023 15:30:59 +0900 Message-Id: <20230330063059.1574380-22-hch@lst.de> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230330063059.1574380-1-hch@lst.de> References: <20230330063059.1574380-1-hch@lst.de> MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org. See http://www.infradead.org/rpr.html Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Most of the code in write_one_subpage_eb and write_one_eb is shared, so merge the two functions into one. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- fs/btrfs/extent_io.c | 77 ++++++++++++++------------------------------ 1 file changed, 25 insertions(+), 52 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index da4ea02a56c8bf..14a8e1ecdf5f16 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1807,53 +1807,11 @@ static void prepare_eb_write(struct extent_buffer *eb) } } -/* - * Unlike the work in write_one_eb(), we rely completely on extent locking. - * Page locking is only utilized at minimum to keep the VMM code happy. - */ -static void write_one_subpage_eb(struct extent_buffer *eb, - struct writeback_control *wbc) -{ - struct btrfs_fs_info *fs_info = eb->fs_info; - struct page *page = eb->pages[0]; - bool no_dirty_ebs = false; - struct btrfs_bio *bbio; - - prepare_eb_write(eb); - - /* clear_page_dirty_for_io() in subpage helper needs page locked */ - lock_page(page); - btrfs_subpage_set_writeback(fs_info, page, eb->start, eb->len); - - /* Check if this is the last dirty bit to update nr_written */ - no_dirty_ebs = btrfs_subpage_clear_and_test_dirty(fs_info, page, - eb->start, eb->len); - if (no_dirty_ebs) - clear_page_dirty_for_io(page); - - bbio = btrfs_bio_alloc(INLINE_EXTENT_BUFFER_PAGES, - REQ_OP_WRITE | REQ_META | wbc_to_write_flags(wbc), - BTRFS_I(eb->fs_info->btree_inode), - extent_buffer_write_end_io, eb); - bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; - bbio->file_offset = eb->start; - __bio_add_page(&bbio->bio, page, eb->len, eb->start - page_offset(page)); - unlock_page(page); - btrfs_submit_bio(bbio, 0); - - /* - * Submission finished without problem, if no range of the page is - * dirty anymore, we have submitted a page. Update nr_written in wbc. - */ - if (no_dirty_ebs) - wbc->nr_to_write--; -} - static noinline_for_stack void write_one_eb(struct extent_buffer *eb, struct writeback_control *wbc) { + struct btrfs_fs_info *fs_info = eb->fs_info; struct btrfs_bio *bbio; - int i, num_pages; prepare_eb_write(eb); @@ -1863,17 +1821,32 @@ static noinline_for_stack void write_one_eb(struct extent_buffer *eb, extent_buffer_write_end_io, eb); bbio->bio.bi_iter.bi_sector = eb->start >> SECTOR_SHIFT; bbio->file_offset = eb->start; - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - struct page *p = eb->pages[i]; + if (fs_info->nodesize < PAGE_SIZE) { + struct page *p = eb->pages[0]; lock_page(p); - clear_page_dirty_for_io(p); - set_page_writeback(p); - __bio_add_page(&bbio->bio, p, PAGE_SIZE, 0); - wbc->nr_to_write--; + btrfs_subpage_set_writeback(fs_info, p, eb->start, eb->len); + if (btrfs_subpage_clear_and_test_dirty(fs_info, p, eb->start, + eb->len)) { + clear_page_dirty_for_io(p); + wbc->nr_to_write--; + } + __bio_add_page(&bbio->bio, p, eb->len, + eb->start - page_offset(p)); unlock_page(p); + } else { + int i; + + for (i = 0; i < num_extent_pages(eb); i++) { + struct page *p = eb->pages[i]; + + lock_page(p); + clear_page_dirty_for_io(p); + set_page_writeback(p); + __bio_add_page(&bbio->bio, p, PAGE_SIZE, 0); + wbc->nr_to_write--; + unlock_page(p); + } } btrfs_submit_bio(bbio, 0); } @@ -1945,7 +1918,7 @@ static int submit_eb_subpage(struct page *page, struct writeback_control *wbc) continue; if (lock_extent_buffer_for_io(eb, wbc)) { - write_one_subpage_eb(eb, wbc); + write_one_eb(eb, wbc); submitted++; } free_extent_buffer(eb);