From patchwork Tue Sep 17 18:43:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17F2B1599 for ; Tue, 17 Sep 2019 18:43:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EA7EE214AF for ; Tue, 17 Sep 2019 18:43:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="QLUZ7PIm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727229AbfIQSnu (ORCPT ); Tue, 17 Sep 2019 14:43:50 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:41453 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727100AbfIQSnt (ORCPT ); Tue, 17 Sep 2019 14:43:49 -0400 Received: by mail-qt1-f195.google.com with SMTP id x4so5686575qtq.8 for ; Tue, 17 Sep 2019 11:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gO+w/U8hMDaFUu707WWbHk/4OoJIRUYRlEDKwVJjTag=; b=QLUZ7PIm+frGdb1GvNxtaDkNyhxh82tNZLrLJvNk+AV2wqO+LCfao30oUlwBjumDk2 zmObSLAG3n695kyOgwUFNDLgXpAOzLbH4JmmdjsEuMVw1rfgiJATAc5p7KAKp1OMnvTb NltaMXe9430uZPxl4jhXApk6cmc/Fsoh4okJ57jtkolFmPLx6WRma0fN5NXH0G5uzK1Z BGMLo37KTEXjZL1I0FXDlf3RJNUsoTqfREcw+/DtnhoYx4GytfpZmhO1LWDJlT1vUFiE uQovvZeRUhMyN+8OYZEAQ6VDtxJxX6rKv7d2zuYWBYOyOY1Dm2BMEcwAfSiFjj+94lbc Bvlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gO+w/U8hMDaFUu707WWbHk/4OoJIRUYRlEDKwVJjTag=; b=IdkTzW4NrgZn2YSKr+8Y1GD9rwjG6O0Ys3PWGRlCcrCQLJTU0xBz0lMo+o6VsHKVQl /xodwvu6fqt6TPYnhfzv8yAd65zaiLr5qI3I00k6fllODscKtTUKlY85XI6B6rdPEohp /x7C1uPSNMSZbnQPDjOs4FvheCQIl8y1PXvGRnyWi7NyQy3BABs1kAOqnZcD45bajN23 hslskafbCh3ITFXbxm9snXUrD1mA7UGVi5cqJxR3KgLTFLHrlhHodngChDFaCcKr9wMc zNIjOUCdZmrvwL8K4VHgcPXN6u+PeybrzC0Lnd0XtLWV3NcisRHSY1siwKONgw7OyqSc rt+A== X-Gm-Message-State: APjAAAX3k/hEc7IfuGjBpl784EaJoBqyl8YprGNxBr4h4wBq/gwncYZq KqPjqCXHbCXl0LsE4RgXlWuryFAN0Qdopw== X-Google-Smtp-Source: APXvYqx9nVGDLB6af1sXAMhqBp2faRs4vQqYqj7/mNlQjehReCvVWXeVkEkoUeM83ZF2dDHJZfedNw== X-Received: by 2002:a0c:f787:: with SMTP id s7mr4317787qvn.221.1568745828563; Tue, 17 Sep 2019 11:43:48 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id r7sm1716174qkb.82.2019.09.17.11.43.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:47 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 1/9] btrfs: separate out the extent leak code Date: Tue, 17 Sep 2019 14:43:35 -0400 Message-Id: <20190917184344.13155-2-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We check both extent buffer and extent state leaks in the same function, separate these two functions out so we can move them around. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 4dc5e6939856..b4b786a9d870 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -60,7 +60,19 @@ void btrfs_leak_debug_del(struct list_head *entry) } static inline -void btrfs_leak_debug_check(void) +void btrfs_extent_buffer_leak_debug_check() +{ + while (!list_empty(&buffers)) { + eb = list_entry(buffers.next, struct extent_buffer, leak_list); + pr_err("BTRFS: buffer leak start %llu len %lu refs %d bflags %lu\n", + eb->start, eb->len, atomic_read(&eb->refs), eb->bflags); + list_del(&eb->leak_list); + kmem_cache_free(extent_buffer_cache, eb); + } +} + +static inline +void btrfs_extent_state_leak_debug_check(void) { struct extent_state *state; struct extent_buffer *eb; @@ -74,14 +86,6 @@ void btrfs_leak_debug_check(void) list_del(&state->leak_list); kmem_cache_free(extent_state_cache, state); } - - while (!list_empty(&buffers)) { - eb = list_entry(buffers.next, struct extent_buffer, leak_list); - pr_err("BTRFS: buffer leak start %llu len %lu refs %d bflags %lu\n", - eb->start, eb->len, atomic_read(&eb->refs), eb->bflags); - list_del(&eb->leak_list); - kmem_cache_free(extent_buffer_cache, eb); - } } #define btrfs_debug_check_extent_io_range(tree, start, end) \ @@ -105,7 +109,8 @@ static inline void __btrfs_debug_check_extent_io_range(const char *caller, #else #define btrfs_leak_debug_add(new, head) do {} while (0) #define btrfs_leak_debug_del(entry) do {} while (0) -#define btrfs_leak_debug_check() do {} while (0) +#define btrfs_extent_buffer_leak_debug_check() do {} while (0) +#define btrfs_extent_state_leak_debug_check() do {} while (0) #define btrfs_debug_check_extent_io_range(c, s, e) do {} while (0) #endif @@ -235,7 +240,8 @@ int __init extent_io_init(void) void __cold extent_io_exit(void) { - btrfs_leak_debug_check(); + btrfs_extent_buffer_leak_debug_check(); + btrfs_extent_state_leak_debug_check(); /* * Make sure all delayed rcu free are flushed before we From patchwork Tue Sep 17 18:43:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B0A214DB for ; Tue, 17 Sep 2019 18:43:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 49A31206C2 for ; Tue, 17 Sep 2019 18:43:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="WDb82yo6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729488AbfIQSnx (ORCPT ); Tue, 17 Sep 2019 14:43:53 -0400 Received: from mail-qk1-f194.google.com ([209.85.222.194]:33066 "EHLO mail-qk1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729422AbfIQSnw (ORCPT ); Tue, 17 Sep 2019 14:43:52 -0400 Received: by mail-qk1-f194.google.com with SMTP id x134so5174153qkb.0 for ; Tue, 17 Sep 2019 11:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vHCP+hHJvToV9YWamNnRTjUFVOTcK8JGrxTKm0NP8ew=; b=WDb82yo6EG5tYsIhNh1BVL9psSHi4meuLNPHEm8hwpLsjG26p+Rg8Pw8Md5BUiVDJq y82IhDjpgrvIHT7E06GUeyH8+pzjL/Z/mmd7a3R+a3AAzKwtxRxBxkaLPfXZtVvykFbK OaU6VxoqyN2HYoOXobCs1dkVga+dwEJXh04duHQk8Ds287hTUcH9MYC9xVQ8mdUuWFuG jVSIi+T7E7YujE31KU/QwNmArPtCBEAyBmpxB+m+JeX8klm0mIBlhMsyXnknEwevwOe7 h3VbbekAMkDT+F4EO1Sd0si1P1NOfJIEHvGbbvI+mLRD1Qg0VGxacPWV5hfssB1lMSaf +UZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vHCP+hHJvToV9YWamNnRTjUFVOTcK8JGrxTKm0NP8ew=; b=owv7Dp1K32VYvGYDVrBA/WFR/KSGGf1MbgfOt5TZbjM5PCCD3pIcde2PL3PghF3FB/ pfcbf/zoNZ0RenCET3FvPiQnAOKVbnTlkdLN75fDOFgHlC4LDGI9OJ+8+Q3JrRHc4vHv a/5xI2kjanybD9OpvP6GKK/fXV/SdADt60xbxxrICHilhoipJmjtr2jJiUEwiP2QiDuv NNDz7DSvuR4MGQBFdC545ips+DcdU9f1e4671LvKz5HAdkVfz3zIpmTaA1ZM+xWvP828 ng2j4ksmML64t6ClwZpZcKa6w8dJpW//ZE/C4FdoMNyUJ5CKo/LNo+HK+7kUDnbHwZKN oLEQ== X-Gm-Message-State: APjAAAWoe9EbhO01uxxMvqCSKEQMuP+kDKtyh1uL/ZeanOnLMa/bBiIi JuYVyjug5iAIUvAf9/6BQzks2+iw+IUe0w== X-Google-Smtp-Source: APXvYqzA0pFqaW3YN4CfY+HFSuxnAbAftM2B9A6G5gloOJnMSsM6XhwSPqQJ+e8Y0d38kOXH8O8HUA== X-Received: by 2002:a37:a704:: with SMTP id q4mr5110546qke.385.1568745830393; Tue, 17 Sep 2019 11:43:50 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id b130sm1758646qkc.100.2019.09.17.11.43.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:49 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 2/9] btrfs: separate out the extent io init function Date: Tue, 17 Sep 2019 14:43:36 -0400 Message-Id: <20190917184344.13155-3-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We are moving extent_io_tree into it's on file, so separate out the extent_state init stuff from extent_io_tree_init(). Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 18 +++++++++++------- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 ++++++++- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index b4b786a9d870..bdd9b7f5a295 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -201,19 +201,23 @@ static int __must_check flush_write_bio(struct extent_page_data *epd) return ret; } -int __init extent_io_init(void) +int __init extent_state_cache_init(void) { extent_state_cache = kmem_cache_create("btrfs_extent_state", sizeof(struct extent_state), 0, SLAB_MEM_SPREAD, NULL); if (!extent_state_cache) return -ENOMEM; + return 0; +} +int __init extent_io_init(void) +{ extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer", sizeof(struct extent_buffer), 0, SLAB_MEM_SPREAD, NULL); if (!extent_buffer_cache) - goto free_state_cache; + return -ENOMEM; if (bioset_init(&btrfs_bioset, BIO_POOL_SIZE, offsetof(struct btrfs_io_bio, bio), @@ -231,24 +235,24 @@ int __init extent_io_init(void) free_buffer_cache: kmem_cache_destroy(extent_buffer_cache); extent_buffer_cache = NULL; + return -ENOMEM; +} -free_state_cache: +void __cold extent_state_cache_exit(void) +{ + btrfs_extent_state_leak_debug_check(); kmem_cache_destroy(extent_state_cache); - extent_state_cache = NULL; - return -ENOMEM; } void __cold extent_io_exit(void) { btrfs_extent_buffer_leak_debug_check(); - btrfs_extent_state_leak_debug_check(); /* * Make sure all delayed rcu free are flushed before we * destroy caches. */ rcu_barrier(); - kmem_cache_destroy(extent_state_cache); kmem_cache_destroy(extent_buffer_cache); bioset_exit(&btrfs_bioset); } diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index cf3424d58fec..e813f593202d 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -556,4 +556,6 @@ bool find_lock_delalloc_range(struct inode *inode, struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, u64 start); +int __init extent_state_cache_init(void); +void __cold extent_state_cache_exit(void); #endif diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 1b151af25772..843015b9a11e 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -2360,10 +2360,14 @@ static int __init init_btrfs_fs(void) if (err) goto free_cachep; - err = extent_map_init(); + err = extent_state_cache_init(); if (err) goto free_extent_io; + err = extent_map_init(); + if (err) + goto free_extent_state_cache; + err = ordered_data_init(); if (err) goto free_extent_map; @@ -2422,6 +2426,8 @@ static int __init init_btrfs_fs(void) ordered_data_exit(); free_extent_map: extent_map_exit(); +free_extent_state_cache: + extent_state_cache_exit(); free_extent_io: extent_io_exit(); free_cachep: @@ -2442,6 +2448,7 @@ static void __exit exit_btrfs_fs(void) btrfs_prelim_ref_exit(); ordered_data_exit(); extent_map_exit(); + extent_state_cache_exit(); extent_io_exit(); btrfs_interface_exit(); btrfs_end_io_wq_exit(); From patchwork Tue Sep 17 18:43:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC75414DB for ; Tue, 17 Sep 2019 18:43:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 86149206C2 for ; Tue, 17 Sep 2019 18:43:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="rtXEQ3Lj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729497AbfIQSnz (ORCPT ); Tue, 17 Sep 2019 14:43:55 -0400 Received: from mail-qk1-f193.google.com ([209.85.222.193]:34728 "EHLO mail-qk1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729422AbfIQSnz (ORCPT ); Tue, 17 Sep 2019 14:43:55 -0400 Received: by mail-qk1-f193.google.com with SMTP id q203so5168165qke.1 for ; Tue, 17 Sep 2019 11:43:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0nOP1bPX4DtUxxN00KaiBrv3aSk3hyVm/FlqiyoiMNc=; b=rtXEQ3Ljp4LXh9S/VGfDYGF4EJ5VjE2iVcTzuUVQ2hWU3Ze4Fw9VbZg9U/JI1uMrL4 7FlUV8oAQgLWCNcl4QH2BeVl76cjAjO4Mtw7YMPivk8UR1lhE2uPtiAEYe9oVK6LJl9z Up2Zzxi0f51eVoGkJZXUq6ii+H1WZGl8EMFQxgIcL8i/jtQ67a751nJ/PoyDFEi5midF 0HoZpkTg+P0k8CDCg/4zqjcTFqVQxGrJWNOTTC/htk5e/EsTH0yPX+XMrl2Yg1pBxyEt Xu8ZQ+rF1VYeS3azxKQwfOg4C0lgGJeoJw8AnmNHNCGodWCWTrZLhgZfhkM6jIsyl6kA /syg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0nOP1bPX4DtUxxN00KaiBrv3aSk3hyVm/FlqiyoiMNc=; b=UKdZ9VDqBHwkD29kxPer7a+EaW+7vyrXqDWtG+2bnz1FYG/+emOWxSjuP2o4C6GTO0 NudQioc/r1EMAFJc7QQPBH8m+Ne3Ei7lFZE/JzzVjqKbpC7LfK9lNlbgg8CyyesWenuP FEERLmRsB+6DXN2zS7HthgBLoVv4L2AL82U6ts7Ud/bFJTyGfvJWXG+PAiBsHgF+uDYU Zzv8/jBTdl/6cyMTig8/q7f4z8ShKLWqlqUSF6kY1DLhTTafcOL9AUHYAFgVx9l3kJi7 Zat1QhzilIQ27NTPIjvJx5lK7LtuI0xsz3ubE4kjGOa1Hx5pHZRjQb6+HinbIq2Ap2uc ojoA== X-Gm-Message-State: APjAAAUEcAqV1cLFsq6ZSlS59iUDPrIcOlvKfZWOHkW8BdNT4DpjdzsX FME4qujMZbBOdXMbmGAvZYVSb5orfMbcmg== X-Google-Smtp-Source: APXvYqzTp03IatsIxbt/TgiKqUDr58hPkOIAmtVSnfx2dVLBJCMNr4W7mKBTgTYy47Jt0WKMBh9FFg== X-Received: by 2002:a37:9ccb:: with SMTP id f194mr5403713qke.150.1568745832184; Tue, 17 Sep 2019 11:43:52 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id n21sm1685223qka.83.2019.09.17.11.43.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:51 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 3/9] btrfs: move extent_io_tree defs to their own header Date: Tue, 17 Sep 2019 14:43:37 -0400 Message-Id: <20190917184344.13155-4-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org extent_io.c/h are huge, encompassing a bunch of different things. The extent_io_tree code can live on its own, so separate this out. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 1 + fs/btrfs/extent-io-tree.h | 227 ++++++++++++++++++++++++++++++++++++++ fs/btrfs/extent_io.c | 1 + fs/btrfs/extent_io.h | 217 +----------------------------------- 4 files changed, 230 insertions(+), 216 deletions(-) create mode 100644 fs/btrfs/extent-io-tree.h diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 19d669d12ca1..5e7ff169683c 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -28,6 +28,7 @@ #include #include #include +#include "extent-io-tree.h" #include "extent_io.h" #include "extent_map.h" #include "async-thread.h" diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h new file mode 100644 index 000000000000..6f53387445ca --- /dev/null +++ b/fs/btrfs/extent-io-tree.h @@ -0,0 +1,227 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef BTRFS_EXTENT_IO_TREE_H +#define BTRFS_EXTENT_IO_TREE_H + +struct extent_changeset; + +/* bits for the extent state */ +#define EXTENT_DIRTY (1U << 0) +#define EXTENT_UPTODATE (1U << 1) +#define EXTENT_LOCKED (1U << 2) +#define EXTENT_NEW (1U << 3) +#define EXTENT_DELALLOC (1U << 4) +#define EXTENT_DEFRAG (1U << 5) +#define EXTENT_BOUNDARY (1U << 6) +#define EXTENT_NODATASUM (1U << 7) +#define EXTENT_CLEAR_META_RESV (1U << 8) +#define EXTENT_NEED_WAIT (1U << 9) +#define EXTENT_DAMAGED (1U << 10) +#define EXTENT_NORESERVE (1U << 11) +#define EXTENT_QGROUP_RESERVED (1U << 12) +#define EXTENT_CLEAR_DATA_RESV (1U << 13) +#define EXTENT_DELALLOC_NEW (1U << 14) +#define EXTENT_DO_ACCOUNTING (EXTENT_CLEAR_META_RESV | \ + EXTENT_CLEAR_DATA_RESV) +#define EXTENT_CTLBITS (EXTENT_DO_ACCOUNTING) + +/* + * Redefined bits above which are used only in the device allocation tree, + * shouldn't be using EXTENT_LOCKED / EXTENT_BOUNDARY / EXTENT_CLEAR_META_RESV + * / EXTENT_CLEAR_DATA_RESV because they have special meaning to the bit + * manipulation functions + */ +#define CHUNK_ALLOCATED EXTENT_DIRTY +#define CHUNK_TRIMMED EXTENT_DEFRAG + +enum { + IO_TREE_FS_INFO_FREED_EXTENTS0, + IO_TREE_FS_INFO_FREED_EXTENTS1, + IO_TREE_INODE_IO, + IO_TREE_INODE_IO_FAILURE, + IO_TREE_RELOC_BLOCKS, + IO_TREE_TRANS_DIRTY_PAGES, + IO_TREE_ROOT_DIRTY_LOG_PAGES, + IO_TREE_SELFTEST, +}; + +struct extent_io_tree { + struct rb_root state; + struct btrfs_fs_info *fs_info; + void *private_data; + u64 dirty_bytes; + bool track_uptodate; + + /* Who owns this io tree, should be one of IO_TREE_* */ + u8 owner; + + spinlock_t lock; + const struct extent_io_ops *ops; +}; + +struct extent_state { + u64 start; + u64 end; /* inclusive */ + struct rb_node rb_node; + + /* ADD NEW ELEMENTS AFTER THIS */ + wait_queue_head_t wq; + refcount_t refs; + unsigned state; + + struct io_failure_record *failrec; + +#ifdef CONFIG_BTRFS_DEBUG + struct list_head leak_list; +#endif +}; + +int __init extent_state_cache_init(void); +void __cold extent_state_cache_exit(void); + +void extent_io_tree_init(struct btrfs_fs_info *fs_info, + struct extent_io_tree *tree, unsigned int owner, + void *private_data); +void extent_io_tree_release(struct extent_io_tree *tree); + +int lock_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, + struct extent_state **cached); + +static inline int lock_extent(struct extent_io_tree *tree, u64 start, u64 end) +{ + return lock_extent_bits(tree, start, end, NULL); +} + +int try_lock_extent(struct extent_io_tree *tree, u64 start, u64 end); + +int __init extent_io_init(void); +void __cold extent_io_exit(void); + +u64 count_range_bits(struct extent_io_tree *tree, + u64 *start, u64 search_end, + u64 max_bytes, unsigned bits, int contig); + +void free_extent_state(struct extent_state *state); +int test_range_bit(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, int filled, + struct extent_state *cached_state); +int clear_record_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, struct extent_changeset *changeset); +int clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, int wake, int delete, + struct extent_state **cached); +int __clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, int wake, int delete, + struct extent_state **cached, gfp_t mask, + struct extent_changeset *changeset); + +static inline int unlock_extent(struct extent_io_tree *tree, u64 start, u64 end) +{ + return clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, NULL); +} + +static inline int unlock_extent_cached(struct extent_io_tree *tree, u64 start, + u64 end, struct extent_state **cached) +{ + return __clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, cached, + GFP_NOFS, NULL); +} + +static inline int unlock_extent_cached_atomic(struct extent_io_tree *tree, + u64 start, u64 end, struct extent_state **cached) +{ + return __clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, cached, + GFP_ATOMIC, NULL); +} + +static inline int clear_extent_bits(struct extent_io_tree *tree, u64 start, + u64 end, unsigned bits) +{ + int wake = 0; + + if (bits & EXTENT_LOCKED) + wake = 1; + + return clear_extent_bit(tree, start, end, bits, wake, 0, NULL); +} + +int set_record_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, struct extent_changeset *changeset); +int set_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, u64 *failed_start, + struct extent_state **cached_state, gfp_t mask); +int set_extent_bits_nowait(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits); + +static inline int set_extent_bits(struct extent_io_tree *tree, u64 start, + u64 end, unsigned bits) +{ + return set_extent_bit(tree, start, end, bits, NULL, NULL, GFP_NOFS); +} + +static inline int clear_extent_uptodate(struct extent_io_tree *tree, u64 start, + u64 end, struct extent_state **cached_state) +{ + return __clear_extent_bit(tree, start, end, EXTENT_UPTODATE, 0, 0, + cached_state, GFP_NOFS, NULL); +} + +static inline int set_extent_dirty(struct extent_io_tree *tree, u64 start, + u64 end, gfp_t mask) +{ + return set_extent_bit(tree, start, end, EXTENT_DIRTY, NULL, + NULL, mask); +} + +static inline int clear_extent_dirty(struct extent_io_tree *tree, u64 start, + u64 end, struct extent_state **cached) +{ + return clear_extent_bit(tree, start, end, + EXTENT_DIRTY | EXTENT_DELALLOC | + EXTENT_DO_ACCOUNTING, 0, 0, cached); +} + +int convert_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, + unsigned bits, unsigned clear_bits, + struct extent_state **cached_state); + +static inline int set_extent_delalloc(struct extent_io_tree *tree, u64 start, + u64 end, unsigned int extra_bits, + struct extent_state **cached_state) +{ + return set_extent_bit(tree, start, end, + EXTENT_DELALLOC | EXTENT_UPTODATE | extra_bits, + NULL, cached_state, GFP_NOFS); +} + +static inline int set_extent_defrag(struct extent_io_tree *tree, u64 start, + u64 end, struct extent_state **cached_state) +{ + return set_extent_bit(tree, start, end, + EXTENT_DELALLOC | EXTENT_UPTODATE | EXTENT_DEFRAG, + NULL, cached_state, GFP_NOFS); +} + +static inline int set_extent_new(struct extent_io_tree *tree, u64 start, + u64 end) +{ + return set_extent_bit(tree, start, end, EXTENT_NEW, NULL, NULL, + GFP_NOFS); +} + +static inline int set_extent_uptodate(struct extent_io_tree *tree, u64 start, + u64 end, struct extent_state **cached_state, gfp_t mask) +{ + return set_extent_bit(tree, start, end, EXTENT_UPTODATE, NULL, + cached_state, mask); +} + +int find_first_extent_bit(struct extent_io_tree *tree, u64 start, + u64 *start_ret, u64 *end_ret, unsigned bits, + struct extent_state **cached_state); +void find_first_clear_extent_bit(struct extent_io_tree *tree, u64 start, + u64 *start_ret, u64 *end_ret, unsigned bits); +int extent_invalidatepage(struct extent_io_tree *tree, + struct page *page, unsigned long offset); + +#endif /* BTRFS_EXTENT_IO_TREE_H */ diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index bdd9b7f5a295..f5945f0a06da 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -14,6 +14,7 @@ #include #include #include "extent_io.h" +#include "extent-io-tree.h" #include "extent_map.h" #include "ctree.h" #include "btrfs_inode.h" diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index e813f593202d..8c782d061132 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -7,35 +7,6 @@ #include #include "ulist.h" -/* bits for the extent state */ -#define EXTENT_DIRTY (1U << 0) -#define EXTENT_UPTODATE (1U << 1) -#define EXTENT_LOCKED (1U << 2) -#define EXTENT_NEW (1U << 3) -#define EXTENT_DELALLOC (1U << 4) -#define EXTENT_DEFRAG (1U << 5) -#define EXTENT_BOUNDARY (1U << 6) -#define EXTENT_NODATASUM (1U << 7) -#define EXTENT_CLEAR_META_RESV (1U << 8) -#define EXTENT_NEED_WAIT (1U << 9) -#define EXTENT_DAMAGED (1U << 10) -#define EXTENT_NORESERVE (1U << 11) -#define EXTENT_QGROUP_RESERVED (1U << 12) -#define EXTENT_CLEAR_DATA_RESV (1U << 13) -#define EXTENT_DELALLOC_NEW (1U << 14) -#define EXTENT_DO_ACCOUNTING (EXTENT_CLEAR_META_RESV | \ - EXTENT_CLEAR_DATA_RESV) -#define EXTENT_CTLBITS (EXTENT_DO_ACCOUNTING) - -/* - * Redefined bits above which are used only in the device allocation tree, - * shouldn't be using EXTENT_LOCKED / EXTENT_BOUNDARY / EXTENT_CLEAR_META_RESV - * / EXTENT_CLEAR_DATA_RESV because they have special meaning to the bit - * manipulation functions - */ -#define CHUNK_ALLOCATED EXTENT_DIRTY -#define CHUNK_TRIMMED EXTENT_DEFRAG - /* * flags for bio submission. The high bits indicate the compression * type for this bio @@ -89,12 +60,11 @@ enum { #define BITMAP_LAST_BYTE_MASK(nbits) \ (BYTE_MASK >> (-(nbits) & (BITS_PER_BYTE - 1))) -struct extent_state; struct btrfs_root; struct btrfs_inode; struct btrfs_io_bio; struct io_failure_record; - +struct extent_io_tree; typedef blk_status_t (extent_submit_bio_start_t)(void *private_data, struct bio *bio, u64 bio_offset); @@ -111,47 +81,6 @@ struct extent_io_ops { int mirror); }; -enum { - IO_TREE_FS_INFO_FREED_EXTENTS0, - IO_TREE_FS_INFO_FREED_EXTENTS1, - IO_TREE_INODE_IO, - IO_TREE_INODE_IO_FAILURE, - IO_TREE_RELOC_BLOCKS, - IO_TREE_TRANS_DIRTY_PAGES, - IO_TREE_ROOT_DIRTY_LOG_PAGES, - IO_TREE_SELFTEST, -}; - -struct extent_io_tree { - struct rb_root state; - struct btrfs_fs_info *fs_info; - void *private_data; - u64 dirty_bytes; - bool track_uptodate; - - /* Who owns this io tree, should be one of IO_TREE_* */ - u8 owner; - - spinlock_t lock; - const struct extent_io_ops *ops; -}; - -struct extent_state { - u64 start; - u64 end; /* inclusive */ - struct rb_node rb_node; - - /* ADD NEW ELEMENTS AFTER THIS */ - wait_queue_head_t wq; - refcount_t refs; - unsigned state; - - struct io_failure_record *failrec; - -#ifdef CONFIG_BTRFS_DEBUG - struct list_head leak_list; -#endif -}; #define INLINE_EXTENT_BUFFER_PAGES 16 #define MAX_INLINE_EXTENT_BUFFER_SIZE (INLINE_EXTENT_BUFFER_PAGES * PAGE_SIZE) @@ -259,152 +188,11 @@ typedef struct extent_map *(get_extent_t)(struct btrfs_inode *inode, u64 start, u64 len, int create); -void extent_io_tree_init(struct btrfs_fs_info *fs_info, - struct extent_io_tree *tree, unsigned int owner, - void *private_data); -void extent_io_tree_release(struct extent_io_tree *tree); int try_release_extent_mapping(struct page *page, gfp_t mask); int try_release_extent_buffer(struct page *page); -int lock_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, - struct extent_state **cached); -static inline int lock_extent(struct extent_io_tree *tree, u64 start, u64 end) -{ - return lock_extent_bits(tree, start, end, NULL); -} - -int try_lock_extent(struct extent_io_tree *tree, u64 start, u64 end); int extent_read_full_page(struct extent_io_tree *tree, struct page *page, get_extent_t *get_extent, int mirror_num); -int __init extent_io_init(void); -void __cold extent_io_exit(void); - -u64 count_range_bits(struct extent_io_tree *tree, - u64 *start, u64 search_end, - u64 max_bytes, unsigned bits, int contig); - -void free_extent_state(struct extent_state *state); -int test_range_bit(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, int filled, - struct extent_state *cached_state); -int clear_record_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, struct extent_changeset *changeset); -int clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, int wake, int delete, - struct extent_state **cached); -int __clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, int wake, int delete, - struct extent_state **cached, gfp_t mask, - struct extent_changeset *changeset); - -static inline int unlock_extent(struct extent_io_tree *tree, u64 start, u64 end) -{ - return clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, NULL); -} - -static inline int unlock_extent_cached(struct extent_io_tree *tree, u64 start, - u64 end, struct extent_state **cached) -{ - return __clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, cached, - GFP_NOFS, NULL); -} - -static inline int unlock_extent_cached_atomic(struct extent_io_tree *tree, - u64 start, u64 end, struct extent_state **cached) -{ - return __clear_extent_bit(tree, start, end, EXTENT_LOCKED, 1, 0, cached, - GFP_ATOMIC, NULL); -} - -static inline int clear_extent_bits(struct extent_io_tree *tree, u64 start, - u64 end, unsigned bits) -{ - int wake = 0; - - if (bits & EXTENT_LOCKED) - wake = 1; - - return clear_extent_bit(tree, start, end, bits, wake, 0, NULL); -} - -int set_record_extent_bits(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, struct extent_changeset *changeset); -int set_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, u64 *failed_start, - struct extent_state **cached_state, gfp_t mask); -int set_extent_bits_nowait(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits); - -static inline int set_extent_bits(struct extent_io_tree *tree, u64 start, - u64 end, unsigned bits) -{ - return set_extent_bit(tree, start, end, bits, NULL, NULL, GFP_NOFS); -} - -static inline int clear_extent_uptodate(struct extent_io_tree *tree, u64 start, - u64 end, struct extent_state **cached_state) -{ - return __clear_extent_bit(tree, start, end, EXTENT_UPTODATE, 0, 0, - cached_state, GFP_NOFS, NULL); -} - -static inline int set_extent_dirty(struct extent_io_tree *tree, u64 start, - u64 end, gfp_t mask) -{ - return set_extent_bit(tree, start, end, EXTENT_DIRTY, NULL, - NULL, mask); -} - -static inline int clear_extent_dirty(struct extent_io_tree *tree, u64 start, - u64 end, struct extent_state **cached) -{ - return clear_extent_bit(tree, start, end, - EXTENT_DIRTY | EXTENT_DELALLOC | - EXTENT_DO_ACCOUNTING, 0, 0, cached); -} - -int convert_extent_bit(struct extent_io_tree *tree, u64 start, u64 end, - unsigned bits, unsigned clear_bits, - struct extent_state **cached_state); - -static inline int set_extent_delalloc(struct extent_io_tree *tree, u64 start, - u64 end, unsigned int extra_bits, - struct extent_state **cached_state) -{ - return set_extent_bit(tree, start, end, - EXTENT_DELALLOC | EXTENT_UPTODATE | extra_bits, - NULL, cached_state, GFP_NOFS); -} - -static inline int set_extent_defrag(struct extent_io_tree *tree, u64 start, - u64 end, struct extent_state **cached_state) -{ - return set_extent_bit(tree, start, end, - EXTENT_DELALLOC | EXTENT_UPTODATE | EXTENT_DEFRAG, - NULL, cached_state, GFP_NOFS); -} - -static inline int set_extent_new(struct extent_io_tree *tree, u64 start, - u64 end) -{ - return set_extent_bit(tree, start, end, EXTENT_NEW, NULL, NULL, - GFP_NOFS); -} - -static inline int set_extent_uptodate(struct extent_io_tree *tree, u64 start, - u64 end, struct extent_state **cached_state, gfp_t mask) -{ - return set_extent_bit(tree, start, end, EXTENT_UPTODATE, NULL, - cached_state, mask); -} - -int find_first_extent_bit(struct extent_io_tree *tree, u64 start, - u64 *start_ret, u64 *end_ret, unsigned bits, - struct extent_state **cached_state); -void find_first_clear_extent_bit(struct extent_io_tree *tree, u64 start, - u64 *start_ret, u64 *end_ret, unsigned bits); -int extent_invalidatepage(struct extent_io_tree *tree, - struct page *page, unsigned long offset); int extent_write_full_page(struct page *page, struct writeback_control *wbc); int extent_write_locked_range(struct inode *inode, u64 start, u64 end, int mode); @@ -555,7 +343,4 @@ bool find_lock_delalloc_range(struct inode *inode, #endif struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, u64 start); - -int __init extent_state_cache_init(void); -void __cold extent_state_cache_exit(void); #endif From patchwork Tue Sep 17 18:43:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4949D1599 for ; Tue, 17 Sep 2019 18:43:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 27D70206C2 for ; Tue, 17 Sep 2019 18:43:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="Bj55mCb7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729512AbfIQSn5 (ORCPT ); Tue, 17 Sep 2019 14:43:57 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:34734 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729422AbfIQSn4 (ORCPT ); Tue, 17 Sep 2019 14:43:56 -0400 Received: by mail-qk1-f195.google.com with SMTP id q203so5168263qke.1 for ; Tue, 17 Sep 2019 11:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1GMqWVBKJc3QOWMl5pxrGiyb3GprPkQHBSR+0/PEQJ4=; b=Bj55mCb7COz8qRloOxjS2fbgsenvfD37QnUIJwltlOm5ZCWUoxk3nUDAPsi6lUA1px EaSb8b7+VgekQvVBtXqqdV/aNMu+zroqHXvlXPZHcJZL/XeqvqEiKXBc1uO9vro4GeY9 g8+LA7uuEm5I6Z31dRUWxbpjweBNmSW7+g6sZT9IW+8V94pMcYzcCbH1VpBujIqj/K6Q UUQCmJKGMxxL03SZOGl9tBWMntGFfoi2AZTx7cKCWs21hfMyohyTE4fInazJ6mnE4kvA oNew3j2Rr6RqE6JEaaN/c2iLTX6mSF5ZuLfhyrPqQKPlvYNwOyQBoXtvfOl+3i0TLisH pofA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1GMqWVBKJc3QOWMl5pxrGiyb3GprPkQHBSR+0/PEQJ4=; b=HtGBMh6+Hw+x3J4OY5WzeLxqIN8heH8vLZYWBjh1P5V8RCAyP1qsIr/Nakbp0LZPYf +39Umm3q84reZO5k/FK7MDstvvb4zPUV3AQrKvYwfXfT18vSnEW500+1b6sLfN78S9Hw Johf1djla+KKpbmpDVzulSMCMnK1qUFcgT3od1MtinStcvBvRMcTC3vD0indYgJEvOXH Kn/ftbtC8eoWfze/CeCozQ6vlgVkZe3TSPSgZD6DmD22Xm4OnyvoRcnqape+6/8HrYFG fMl3zeSmIuJ1prt6MnP7FFJye+FyFIbay1PmyPAxsdPFjs4NDDbTN8ZaPaTRVCyY4BC1 unyA== X-Gm-Message-State: APjAAAVnpQxTyl6jzDRbD0P+4JZcJY5iBAvphRviBHLUjDIJwOTHcQtT jz2Sm++eylX0coUzmGILOrrPn7RLj4MVsg== X-Google-Smtp-Source: APXvYqy1amvOuoe9WuXHUSYQpCTUfaKClIljy9ZNbXW1VaQNVUDe+32bwaKfX4reWG70/uxkc3tz/A== X-Received: by 2002:ae9:e902:: with SMTP id x2mr5537500qkf.293.1568745833950; Tue, 17 Sep 2019 11:43:53 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id a190sm1830381qkf.118.2019.09.17.11.43.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:53 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 4/9] btrfs: export find_delalloc_range Date: Tue, 17 Sep 2019 14:43:38 -0400 Message-Id: <20190917184344.13155-5-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This utilizes internal stuff to the extent_io_tree, so we need to export it before we move it. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 3 +++ fs/btrfs/extent_io.c | 10 +++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h index 6f53387445ca..02fbb0d4fa88 100644 --- a/fs/btrfs/extent-io-tree.h +++ b/fs/btrfs/extent-io-tree.h @@ -223,5 +223,8 @@ void find_first_clear_extent_bit(struct extent_io_tree *tree, u64 start, u64 *start_ret, u64 *end_ret, unsigned bits); int extent_invalidatepage(struct extent_io_tree *tree, struct page *page, unsigned long offset); +bool btrfs_find_delalloc_range(struct extent_io_tree *tree, u64 *start, + u64 *end, u64 max_bytes, + struct extent_state **cached_state); #endif /* BTRFS_EXTENT_IO_TREE_H */ diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index f5945f0a06da..c47cc2586b37 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1687,9 +1687,9 @@ void find_first_clear_extent_bit(struct extent_io_tree *tree, u64 start, * * true is returned if we find something, false if nothing was in the tree */ -static noinline bool find_delalloc_range(struct extent_io_tree *tree, - u64 *start, u64 *end, u64 max_bytes, - struct extent_state **cached_state) +bool btrfs_find_delalloc_range(struct extent_io_tree *tree, u64 *start, + u64 *end, u64 max_bytes, + struct extent_state **cached_state) { struct rb_node *node; struct extent_state *state; @@ -1807,8 +1807,8 @@ noinline_for_stack bool find_lock_delalloc_range(struct inode *inode, /* step one, find a bunch of delalloc bytes starting at start */ delalloc_start = *start; delalloc_end = 0; - found = find_delalloc_range(tree, &delalloc_start, &delalloc_end, - max_bytes, &cached_state); + found = btrfs_find_delalloc_range(tree, &delalloc_start, &delalloc_end, + max_bytes, &cached_state); if (!found || delalloc_end <= *start) { *start = delalloc_start; *end = delalloc_end; From patchwork Tue Sep 17 18:43:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D27B1599 for ; Tue, 17 Sep 2019 18:43:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6B1AA206C2 for ; Tue, 17 Sep 2019 18:43:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="rwA0L9Rw" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729514AbfIQSn6 (ORCPT ); Tue, 17 Sep 2019 14:43:58 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:38497 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729501AbfIQSn6 (ORCPT ); Tue, 17 Sep 2019 14:43:58 -0400 Received: by mail-qt1-f193.google.com with SMTP id j31so5721018qta.5 for ; Tue, 17 Sep 2019 11:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=q3yCz3OWSJKt06THdJjHiLFdaX0q0cm8at0obtbzLB4=; b=rwA0L9RwNm5Dz6EMDmD1THUMiJ5FvlNmk/fFLpXZ0IIvqs+F/+bPkn3x48CeZPRBO0 DF2H8xEpA524dty6IFXv/rktlHwU0oNfMi9m2AU4Ai7QIWVUQ6ICJ5lIxtHxJ59FZwkC GFVjX4vsjvgjkcPdvYuW9JuKt0e2/opI37o0OL//1yL9w4yciHXTvaWOY94XPWvV43w5 4+NxWV/KK1jAyHa7RojN2/UDTR1NbteY+b6UNcVOX7LiSRg/S56QEpz+WD+6Mcshfxbg l180ahiKn9RPNxhbUF7YtWKXn4cyUU6ZPoUmO7chi7dqIIHQTO+JcWvbwvBViQLatmks d7Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q3yCz3OWSJKt06THdJjHiLFdaX0q0cm8at0obtbzLB4=; b=Z8jZ80WBdkN661T2whWBMRU1Sm21Uh1siBF4fLGTPxMd/c+aYWB6yfPyDBe2RF7kiK MojTlBaFiY6k9zvtRzCFZRx8o17nsOwf/CO2H4OKN1lgysSwaNlZ7p6tCCQT+BIOQlxG Z+/1UD63Nr/lBj5zckeER1PGx0BuZnyd5i3OdA96vA9BqGIyR/ARUDI77QsGsFa+bO/V kPC2k0qDflddn6RHp3FXrA8OaWE90mw+UaKE4ANLhYBYHBe0E8VQicUzzi+Q9+DXp/W+ i7mw4OxguZvb4Pujr4ueiBb+GBPUi/MY3SYIq5f7kQxIen9gsoS+BZA/IMLQOUY5swTG I4/g== X-Gm-Message-State: APjAAAV5qPr/MJFRDt62jvSZZNc/Y4/JwLs0uHxBx0Gcb4rt8EJbDw/C X4uXCQuYLxSqSyfohvz0b72G4JMBlbnvcQ== X-Google-Smtp-Source: APXvYqxWWvwbE7WyY/oeQyUnRrIUVvgVmjavspmbniZFXg8UAU7U3t6Mveh8bgFSLn6h3+RirhHjWA== X-Received: by 2002:ac8:2eaa:: with SMTP id h39mr166015qta.389.1568745835889; Tue, 17 Sep 2019 11:43:55 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u27sm337737qta.90.2019.09.17.11.43.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:55 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 5/9] btrfs: move the failrec tree stuff into extent-io-tree.h Date: Tue, 17 Sep 2019 14:43:39 -0400 Message-Id: <20190917184344.13155-6-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This needs to be cleaned up in the future, but for now it belongs to the extent-io-tree stuff since it uses the internal tree search code. Needed to export get_state_failrec and set_state_failrec as well since we're not going to move the actual IO part of the failrec stuff out at this point. Signed-off-by: Josef Bacik --- fs/btrfs/extent-io-tree.h | 18 ++++++++++++++++++ fs/btrfs/extent_io.c | 8 ++++---- fs/btrfs/extent_io.h | 11 ----------- 3 files changed, 22 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/extent-io-tree.h b/fs/btrfs/extent-io-tree.h index 02fbb0d4fa88..0ba3222ee1f0 100644 --- a/fs/btrfs/extent-io-tree.h +++ b/fs/btrfs/extent-io-tree.h @@ -4,6 +4,7 @@ #define BTRFS_EXTENT_IO_TREE_H struct extent_changeset; +struct io_failure_record; /* bits for the extent state */ #define EXTENT_DIRTY (1U << 0) @@ -227,4 +228,21 @@ bool btrfs_find_delalloc_range(struct extent_io_tree *tree, u64 *start, u64 *end, u64 max_bytes, struct extent_state **cached_state); +/* This should be reworked in the future and put elsewhere. */ +int get_state_failrec(struct extent_io_tree *tree, u64 start, + struct io_failure_record **failrec); +int set_state_failrec(struct extent_io_tree *tree, u64 start, + struct io_failure_record *failrec); +void btrfs_free_io_failure_record(struct btrfs_inode *inode, u64 start, + u64 end); +int btrfs_get_io_failure_record(struct inode *inode, u64 start, u64 end, + struct io_failure_record **failrec_ret); +int free_io_failure(struct extent_io_tree *failure_tree, + struct extent_io_tree *io_tree, + struct io_failure_record *rec); +int clean_io_failure(struct btrfs_fs_info *fs_info, + struct extent_io_tree *failure_tree, + struct extent_io_tree *io_tree, u64 start, + struct page *page, u64 ino, unsigned int pg_offset); + #endif /* BTRFS_EXTENT_IO_TREE_H */ diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index c47cc2586b37..f8134e319b60 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -2025,8 +2025,8 @@ u64 count_range_bits(struct extent_io_tree *tree, * set the private field for a given byte offset in the tree. If there isn't * an extent_state there already, this does nothing. */ -static noinline int set_state_failrec(struct extent_io_tree *tree, u64 start, - struct io_failure_record *failrec) +int set_state_failrec(struct extent_io_tree *tree, u64 start, + struct io_failure_record *failrec) { struct rb_node *node; struct extent_state *state; @@ -2053,8 +2053,8 @@ static noinline int set_state_failrec(struct extent_io_tree *tree, u64 start, return ret; } -static noinline int get_state_failrec(struct extent_io_tree *tree, u64 start, - struct io_failure_record **failrec) +int get_state_failrec(struct extent_io_tree *tree, u64 start, + struct io_failure_record **failrec) { struct rb_node *node; struct extent_state *state; diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 8c782d061132..e22045cef89b 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -296,10 +296,6 @@ struct btrfs_inode; int repair_io_failure(struct btrfs_fs_info *fs_info, u64 ino, u64 start, u64 length, u64 logical, struct page *page, unsigned int pg_offset, int mirror_num); -int clean_io_failure(struct btrfs_fs_info *fs_info, - struct extent_io_tree *failure_tree, - struct extent_io_tree *io_tree, u64 start, - struct page *page, u64 ino, unsigned int pg_offset); void end_extent_writepage(struct page *page, int err, u64 start, u64 end); int btrfs_repair_eb_io_failure(struct extent_buffer *eb, int mirror_num); @@ -323,19 +319,12 @@ struct io_failure_record { }; -void btrfs_free_io_failure_record(struct btrfs_inode *inode, u64 start, - u64 end); -int btrfs_get_io_failure_record(struct inode *inode, u64 start, u64 end, - struct io_failure_record **failrec_ret); bool btrfs_check_repairable(struct inode *inode, unsigned failed_bio_pages, struct io_failure_record *failrec, int fail_mirror); struct bio *btrfs_create_repair_bio(struct inode *inode, struct bio *failed_bio, struct io_failure_record *failrec, struct page *page, int pg_offset, int icsum, bio_end_io_t *endio_func, void *data); -int free_io_failure(struct extent_io_tree *failure_tree, - struct extent_io_tree *io_tree, - struct io_failure_record *rec); #ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS bool find_lock_delalloc_range(struct inode *inode, struct page *locked_page, u64 *start, From patchwork Tue Sep 17 18:43:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1B4B514DB for ; Tue, 17 Sep 2019 18:44:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EE34C206C2 for ; Tue, 17 Sep 2019 18:44:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="OnhWXRxj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729529AbfIQSoD (ORCPT ); Tue, 17 Sep 2019 14:44:03 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:43443 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729311AbfIQSoC (ORCPT ); Tue, 17 Sep 2019 14:44:02 -0400 Received: by mail-qk1-f196.google.com with SMTP id h126so5103820qke.10 for ; Tue, 17 Sep 2019 11:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MD6dLqCOt6gvNUDZYBcbdX7V6O6ilKUHTe5sTQPtsR8=; b=OnhWXRxjVRChoqTkYfiPR3qXdJugXG6zVhcx8zLKxWZCWDZQKvO8F0FI1qOf30JsDh dbct5UQ0bxeSI7rVbegWhLzOGKkUFVnmJk8e95pBSwkk22Ds9jTOUI1Pu5fAz0iD5meb x2n9G1gvKh285B+osp+bLK1iEeu/GdRxJAeuAZXAfbdo9mKLuOtYQGiyiiA8+y/mJz2Q uO9c9NcSt0HWoF30Qa4mldhh2XsoyKpkIvAiwcuk1AstVaSV5tczJ683c7Fi9a83EFQz dPctGs8jwNa6XmimgHaNQnOYOund6NV7vIe4fcX/mUGYlv/yK+7/9UuNqgktClqfFjIA kN6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MD6dLqCOt6gvNUDZYBcbdX7V6O6ilKUHTe5sTQPtsR8=; b=snfnncTBF6FekonbhLam0XvpfJDEojL1zwwM5iWUFl2V2Fz4NGtisD7uB/PG57tkW6 FWo/JCJOSxzqQMeQy5XWWrJSZuXIEdujA8JLkJyrcDUoqeeFUu47tJ1hsRwhB1Yr127a BtiRv9zuknq5RrbxxIhuFoKVh+iQavxCYCNQpYa+EXBBaWG3Pp/eNzzlasDivD/Oyx+D rF3jtHl4U8S714AntiVk+g+B/1zxaGmyhDgKqmJTIFt6ci1rGk+TlxXyVOiD18xHSPXE Z6jJwhhCdh/ReY75WMAPSiJsLn1hsB0qv6pKem+2EIdDJSr7CKxO54OD8dmiEp2qrz9B YDVA== X-Gm-Message-State: APjAAAU3aff/qMgR+mn+rjKkgeDMIlUQHoCLRv0foGJ4RNTzS0VGOlz4 j8Duwt+mNrNwzuIKgyzJ+GCYPJ6qZploQg== X-Google-Smtp-Source: APXvYqzYfgSaEwyIIE+d90EwHifOIpuF+g6BoKnvi84GrleZbe5bwX7MTDLEWPV1Ly1fwbeMvOKwSQ== X-Received: by 2002:ae9:d803:: with SMTP id u3mr5175247qkf.131.1568745839820; Tue, 17 Sep 2019 11:43:59 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id c126sm1608711qkd.13.2019.09.17.11.43.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:43:59 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 7/9] btrfs: separate out the extent buffer init code Date: Tue, 17 Sep 2019 14:43:41 -0400 Message-Id: <20190917184344.13155-8-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org We want to move this into it's own file, so separate out the init/exit code for setting up the extent_buffer cache. Signed-off-by: Josef Bacik --- fs/btrfs/extent_io.c | 36 +++++++++++++++++++----------------- fs/btrfs/extent_io.h | 2 ++ fs/btrfs/super.c | 9 ++++++++- 3 files changed, 29 insertions(+), 18 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index d01d7575f0e9..db85c79e2f90 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -135,32 +135,30 @@ static int __must_check flush_write_bio(struct extent_page_data *epd) int __init extent_io_init(void) { - extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer", - sizeof(struct extent_buffer), 0, - SLAB_MEM_SPREAD, NULL); - if (!extent_buffer_cache) - return -ENOMEM; - if (bioset_init(&btrfs_bioset, BIO_POOL_SIZE, offsetof(struct btrfs_io_bio, bio), BIOSET_NEED_BVECS)) - goto free_buffer_cache; + return -ENOMEM; - if (bioset_integrity_create(&btrfs_bioset, BIO_POOL_SIZE)) - goto free_bioset; + if (bioset_integrity_create(&btrfs_bioset, BIO_POOL_SIZE)) { + bioset_exit(&btrfs_bioset); + return -ENOMEM; + } return 0; +} -free_bioset: - bioset_exit(&btrfs_bioset); - -free_buffer_cache: - kmem_cache_destroy(extent_buffer_cache); - extent_buffer_cache = NULL; - return -ENOMEM; +int __init extent_buffer_init(void) +{ + extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer", + sizeof(struct extent_buffer), 0, + SLAB_MEM_SPREAD, NULL); + if (!extent_buffer_cache) + return -ENOMEM; + return 0; } -void __cold extent_io_exit(void) +void __cold extent_buffer_exit(void) { btrfs_extent_buffer_leak_debug_check(); @@ -170,6 +168,10 @@ void __cold extent_io_exit(void) */ rcu_barrier(); kmem_cache_destroy(extent_buffer_cache); +} + +void __cold extent_io_exit(void) +{ bioset_exit(&btrfs_bioset); } diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index e22045cef89b..e2246956e544 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -332,4 +332,6 @@ bool find_lock_delalloc_range(struct inode *inode, #endif struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, u64 start); +int __init extent_buffer_init(void); +void __cold extent_buffer_exit(void); #endif diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index 843015b9a11e..7207bb39f236 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -2360,10 +2360,14 @@ static int __init init_btrfs_fs(void) if (err) goto free_cachep; - err = extent_state_cache_init(); + err = extent_buffer_init(); if (err) goto free_extent_io; + err = extent_state_cache_init(); + if (err) + goto free_extent_buffer; + err = extent_map_init(); if (err) goto free_extent_state_cache; @@ -2428,6 +2432,8 @@ static int __init init_btrfs_fs(void) extent_map_exit(); free_extent_state_cache: extent_state_cache_exit(); +free_extent_buffer: + extent_buffer_exit(); free_extent_io: extent_io_exit(); free_cachep: @@ -2449,6 +2455,7 @@ static void __exit exit_btrfs_fs(void) ordered_data_exit(); extent_map_exit(); extent_state_cache_exit(); + extent_buffer_exit(); extent_io_exit(); btrfs_interface_exit(); btrfs_end_io_wq_exit(); From patchwork Tue Sep 17 18:43:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2E321599 for ; Tue, 17 Sep 2019 18:44:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9F65C206C2 for ; Tue, 17 Sep 2019 18:44:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="KkCav0Rp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729531AbfIQSoE (ORCPT ); Tue, 17 Sep 2019 14:44:04 -0400 Received: from mail-qk1-f196.google.com ([209.85.222.196]:36824 "EHLO mail-qk1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729501AbfIQSoE (ORCPT ); Tue, 17 Sep 2019 14:44:04 -0400 Received: by mail-qk1-f196.google.com with SMTP id y189so14101qkc.3 for ; Tue, 17 Sep 2019 11:44:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fNzTjzeQY0QFU3S7mmuwxZfDcXCCVbDfHFYUaatK8aU=; b=KkCav0Rpz0g2h5/gjIQMnQ1/0Z1bT2BIYBNg9MFNQ/5H/O/3yydwal6UWLq4daF0/c H6qemT5IUvCKTHlc0of5KTrKX2AUu5z5sqkyDYvn9mBQKC089t6cR+Ei2XSQVXH6HaHR TSOdXHLgbnHmfh+gG3TXMKyxKmvX4oExyjEWFeHVB/AABEbzb7dYg/vHJMTTElETjfL0 DAtomn9STUY3uv3MCGAgZxdItstRbl4Fl+kFuujp5Fn/Do+LTAR5iBk1ZOw/CBKqK5Be mV7PSj5t3j7y/UN6N8p7CjPkQmJu4UwLc5O7yQiBSIYWezlPPHMlHAHvTawAP7gJyMzK no5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fNzTjzeQY0QFU3S7mmuwxZfDcXCCVbDfHFYUaatK8aU=; b=TKwjziXycqpWoDEZLHLy4uByJOspB9ae88+7gTi8hyzicZIH2lVlQvw0mBhFV9rsAY NBxQR/sam8gJDEhaW9QNcr20lODns+eFPh1eJC4msLuBGWHqaEJqSqTWHrMREDkqoQ6r URkWjd1rbJqx5FOoCk8zNmYv9LXtQGR8Ka/3V6+SM1YjH5prwPdhRe4JmJZKHclzaQTU x5ODaZWaofKPYmrYyqmiUetir/knX/AOKZQTsN1+Qg1fJ0fCal32A+6atfrd8eD9xF/r gh54iozIGhNFgJVp/1IAB6qCMbY9oTV8VUO/1Xb1oxIHkydmeISRC9AxAPIHZ76Dqf2e P4pA== X-Gm-Message-State: APjAAAXn3zynVw8bDG6F7E4e8XO/LCgNwj2tdQdoKRuyRQ2LHkUX9vDv 9ag7GoULdm4RSci2NsaL05rOKR581FOOTQ== X-Google-Smtp-Source: APXvYqwLKQpZU7xnKrXBXe/NQnq0PZdo5JyShhKAmx6J8YEGVaaT7DpEbACMgDmp4mZN0sUi3AczbA== X-Received: by 2002:a37:4cc7:: with SMTP id z190mr5447140qka.458.1568745841561; Tue, 17 Sep 2019 11:44:01 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id v12sm2038402qtb.5.2019.09.17.11.44.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:44:00 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 8/9] btrfs: migrate the extent_buffer code out of extent-io.h Date: Tue, 17 Sep 2019 14:43:42 -0400 Message-Id: <20190917184344.13155-9-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org There's two main things we do with extent buffers, we mess with the buffers themselves, and then we do IO on those buffers. Separate out the actual extent buffer manipulation code from extent_io since they are unrelated. Signed-off-by: Josef Bacik --- fs/btrfs/ctree.h | 2 +- fs/btrfs/disk-io.h | 2 + fs/btrfs/extent-buffer.h | 152 +++++++++++++++++++++++++++++++++++++++ fs/btrfs/extent_io.h | 148 +------------------------------------- 4 files changed, 157 insertions(+), 147 deletions(-) create mode 100644 fs/btrfs/extent-buffer.h diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5e7ff169683c..c3a7df64db32 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -29,7 +29,7 @@ #include #include #include "extent-io-tree.h" -#include "extent_io.h" +#include "extent-buffer.h" #include "extent_map.h" #include "async-thread.h" #include "block-rsv.h" diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h index a6958103d87e..affb45b1ed82 100644 --- a/fs/btrfs/disk-io.h +++ b/fs/btrfs/disk-io.h @@ -6,6 +6,8 @@ #ifndef BTRFS_DISK_IO_H #define BTRFS_DISK_IO_H +#include "extent_io.h" + #define BTRFS_SUPER_INFO_OFFSET SZ_64K #define BTRFS_SUPER_INFO_SIZE 4096 diff --git a/fs/btrfs/extent-buffer.h b/fs/btrfs/extent-buffer.h new file mode 100644 index 000000000000..8bb6b695b70f --- /dev/null +++ b/fs/btrfs/extent-buffer.h @@ -0,0 +1,152 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef BTRFS_EXTENT_BUFFER_H +#define BTRFS_EXTENT_BUFFER_H + +enum { + EXTENT_BUFFER_UPTODATE, + EXTENT_BUFFER_DIRTY, + EXTENT_BUFFER_CORRUPT, + /* this got triggered by readahead */ + EXTENT_BUFFER_READAHEAD, + EXTENT_BUFFER_TREE_REF, + EXTENT_BUFFER_STALE, + EXTENT_BUFFER_WRITEBACK, + /* read IO error */ + EXTENT_BUFFER_READ_ERR, + EXTENT_BUFFER_UNMAPPED, + EXTENT_BUFFER_IN_TREE, + /* write IO error */ + EXTENT_BUFFER_WRITE_ERR, +}; + +#define INLINE_EXTENT_BUFFER_PAGES 16 +#define MAX_INLINE_EXTENT_BUFFER_SIZE (INLINE_EXTENT_BUFFER_PAGES * PAGE_SIZE) +struct extent_buffer { + u64 start; + unsigned long len; + unsigned long bflags; + struct btrfs_fs_info *fs_info; + spinlock_t refs_lock; + atomic_t refs; + atomic_t io_pages; + int read_mirror; + struct rcu_head rcu_head; + pid_t lock_owner; + + int blocking_writers; + atomic_t blocking_readers; + bool lock_nested; + /* >= 0 if eb belongs to a log tree, -1 otherwise */ + short log_index; + + /* protects write locks */ + rwlock_t lock; + + /* readers use lock_wq while they wait for the write + * lock holders to unlock + */ + wait_queue_head_t write_lock_wq; + + /* writers use read_lock_wq while they wait for readers + * to unlock + */ + wait_queue_head_t read_lock_wq; + struct page *pages[INLINE_EXTENT_BUFFER_PAGES]; +#ifdef CONFIG_BTRFS_DEBUG + int spinning_writers; + atomic_t spinning_readers; + atomic_t read_locks; + int write_locks; + struct list_head leak_list; +#endif +}; + +/* + * The extent buffer bitmap operations are done with byte granularity instead of + * word granularity for two reasons: + * 1. The bitmaps must be little-endian on disk. + * 2. Bitmap items are not guaranteed to be aligned to a word and therefore a + * single word in a bitmap may straddle two pages in the extent buffer. + */ +#define BIT_BYTE(nr) ((nr) / BITS_PER_BYTE) +#define BYTE_MASK ((1 << BITS_PER_BYTE) - 1) +#define BITMAP_FIRST_BYTE_MASK(start) \ + ((BYTE_MASK << ((start) & (BITS_PER_BYTE - 1))) & BYTE_MASK) +#define BITMAP_LAST_BYTE_MASK(nbits) \ + (BYTE_MASK >> (-(nbits) & (BITS_PER_BYTE - 1))) + +int __init extent_buffer_init(void); +void __cold extent_buffer_exit(void); +int try_release_extent_buffer(struct page *page); +struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start); +struct extent_buffer *__alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start, unsigned long len); +struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start); +struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start); +struct extent_buffer *btrfs_clone_extent_buffer(struct extent_buffer *src); +struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start); +void free_extent_buffer(struct extent_buffer *eb); +void free_extent_buffer_stale(struct extent_buffer *eb); + +void wait_on_extent_buffer_writeback(struct extent_buffer *eb); + +static inline int num_extent_pages(const struct extent_buffer *eb) +{ + return (round_up(eb->start + eb->len, PAGE_SIZE) >> PAGE_SHIFT) - + (eb->start >> PAGE_SHIFT); +} + +static inline void extent_buffer_get(struct extent_buffer *eb) +{ + atomic_inc(&eb->refs); +} + +static inline int extent_buffer_uptodate(struct extent_buffer *eb) +{ + return test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); +} + +int memcmp_extent_buffer(const struct extent_buffer *eb, const void *ptrv, + unsigned long start, unsigned long len); +void read_extent_buffer(const struct extent_buffer *eb, void *dst, + unsigned long start, + unsigned long len); +int read_extent_buffer_to_user(const struct extent_buffer *eb, + void __user *dst, unsigned long start, + unsigned long len); +void write_extent_buffer_fsid(struct extent_buffer *eb, const void *src); +void write_extent_buffer_chunk_tree_uuid(struct extent_buffer *eb, + const void *src); +void write_extent_buffer(struct extent_buffer *eb, const void *src, + unsigned long start, unsigned long len); +void copy_extent_buffer_full(struct extent_buffer *dst, + struct extent_buffer *src); +void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src, + unsigned long dst_offset, unsigned long src_offset, + unsigned long len); +void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, + unsigned long src_offset, unsigned long len); +void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, + unsigned long src_offset, unsigned long len); +void memzero_extent_buffer(struct extent_buffer *eb, unsigned long start, + unsigned long len); +int extent_buffer_test_bit(struct extent_buffer *eb, unsigned long start, + unsigned long pos); +void extent_buffer_bitmap_set(struct extent_buffer *eb, unsigned long start, + unsigned long pos, unsigned long len); +void extent_buffer_bitmap_clear(struct extent_buffer *eb, unsigned long start, + unsigned long pos, unsigned long len); +void clear_extent_buffer_dirty(struct extent_buffer *eb); +bool set_extent_buffer_dirty(struct extent_buffer *eb); +void set_extent_buffer_uptodate(struct extent_buffer *eb); +void clear_extent_buffer_uptodate(struct extent_buffer *eb); +int extent_buffer_under_io(struct extent_buffer *eb); +int map_private_extent_buffer(const struct extent_buffer *eb, + unsigned long offset, unsigned long min_len, + char **map, unsigned long *map_start, + unsigned long *map_len); +#endif /* BTRFS_EXTENT_BUFFER_H */ diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index e2246956e544..c329eb3f1817 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -14,23 +14,6 @@ #define EXTENT_BIO_COMPRESSED 1 #define EXTENT_BIO_FLAG_SHIFT 16 -enum { - EXTENT_BUFFER_UPTODATE, - EXTENT_BUFFER_DIRTY, - EXTENT_BUFFER_CORRUPT, - /* this got triggered by readahead */ - EXTENT_BUFFER_READAHEAD, - EXTENT_BUFFER_TREE_REF, - EXTENT_BUFFER_STALE, - EXTENT_BUFFER_WRITEBACK, - /* read IO error */ - EXTENT_BUFFER_READ_ERR, - EXTENT_BUFFER_UNMAPPED, - EXTENT_BUFFER_IN_TREE, - /* write IO error */ - EXTENT_BUFFER_WRITE_ERR, -}; - /* these are flags for __process_pages_contig */ #define PAGE_UNLOCK (1 << 0) #define PAGE_CLEAR_DIRTY (1 << 1) @@ -46,25 +29,12 @@ enum { */ #define EXTENT_PAGE_PRIVATE 1 -/* - * The extent buffer bitmap operations are done with byte granularity instead of - * word granularity for two reasons: - * 1. The bitmaps must be little-endian on disk. - * 2. Bitmap items are not guaranteed to be aligned to a word and therefore a - * single word in a bitmap may straddle two pages in the extent buffer. - */ -#define BIT_BYTE(nr) ((nr) / BITS_PER_BYTE) -#define BYTE_MASK ((1 << BITS_PER_BYTE) - 1) -#define BITMAP_FIRST_BYTE_MASK(start) \ - ((BYTE_MASK << ((start) & (BITS_PER_BYTE - 1))) & BYTE_MASK) -#define BITMAP_LAST_BYTE_MASK(nbits) \ - (BYTE_MASK >> (-(nbits) & (BITS_PER_BYTE - 1))) - struct btrfs_root; struct btrfs_inode; struct btrfs_io_bio; struct io_failure_record; struct extent_io_tree; +struct extent_buffer; typedef blk_status_t (extent_submit_bio_start_t)(void *private_data, struct bio *bio, u64 bio_offset); @@ -81,49 +51,6 @@ struct extent_io_ops { int mirror); }; - -#define INLINE_EXTENT_BUFFER_PAGES 16 -#define MAX_INLINE_EXTENT_BUFFER_SIZE (INLINE_EXTENT_BUFFER_PAGES * PAGE_SIZE) -struct extent_buffer { - u64 start; - unsigned long len; - unsigned long bflags; - struct btrfs_fs_info *fs_info; - spinlock_t refs_lock; - atomic_t refs; - atomic_t io_pages; - int read_mirror; - struct rcu_head rcu_head; - pid_t lock_owner; - - int blocking_writers; - atomic_t blocking_readers; - bool lock_nested; - /* >= 0 if eb belongs to a log tree, -1 otherwise */ - short log_index; - - /* protects write locks */ - rwlock_t lock; - - /* readers use lock_wq while they wait for the write - * lock holders to unlock - */ - wait_queue_head_t write_lock_wq; - - /* writers use read_lock_wq while they wait for readers - * to unlock - */ - wait_queue_head_t read_lock_wq; - struct page *pages[INLINE_EXTENT_BUFFER_PAGES]; -#ifdef CONFIG_BTRFS_DEBUG - int spinning_writers; - atomic_t spinning_readers; - atomic_t read_locks; - int write_locks; - struct list_head leak_list; -#endif -}; - /* * Structure to record how many bytes and which ranges are set/cleared */ @@ -189,7 +116,6 @@ typedef struct extent_map *(get_extent_t)(struct btrfs_inode *inode, int create); int try_release_extent_mapping(struct page *page, gfp_t mask); -int try_release_extent_buffer(struct page *page); int extent_read_full_page(struct extent_io_tree *tree, struct page *page, get_extent_t *get_extent, int mirror_num); @@ -206,79 +132,13 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, __u64 start, __u64 len); void set_page_extent_mapped(struct page *page); -struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start); -struct extent_buffer *__alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start, unsigned long len); -struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start); -struct extent_buffer *btrfs_clone_extent_buffer(struct extent_buffer *src); -struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start); -void free_extent_buffer(struct extent_buffer *eb); -void free_extent_buffer_stale(struct extent_buffer *eb); #define WAIT_NONE 0 #define WAIT_COMPLETE 1 #define WAIT_PAGE_LOCK 2 + int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num); -void wait_on_extent_buffer_writeback(struct extent_buffer *eb); - -static inline int num_extent_pages(const struct extent_buffer *eb) -{ - return (round_up(eb->start + eb->len, PAGE_SIZE) >> PAGE_SHIFT) - - (eb->start >> PAGE_SHIFT); -} - -static inline void extent_buffer_get(struct extent_buffer *eb) -{ - atomic_inc(&eb->refs); -} - -static inline int extent_buffer_uptodate(struct extent_buffer *eb) -{ - return test_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); -} -int memcmp_extent_buffer(const struct extent_buffer *eb, const void *ptrv, - unsigned long start, unsigned long len); -void read_extent_buffer(const struct extent_buffer *eb, void *dst, - unsigned long start, - unsigned long len); -int read_extent_buffer_to_user(const struct extent_buffer *eb, - void __user *dst, unsigned long start, - unsigned long len); -void write_extent_buffer_fsid(struct extent_buffer *eb, const void *src); -void write_extent_buffer_chunk_tree_uuid(struct extent_buffer *eb, - const void *src); -void write_extent_buffer(struct extent_buffer *eb, const void *src, - unsigned long start, unsigned long len); -void copy_extent_buffer_full(struct extent_buffer *dst, - struct extent_buffer *src); -void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src, - unsigned long dst_offset, unsigned long src_offset, - unsigned long len); -void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, - unsigned long src_offset, unsigned long len); -void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, - unsigned long src_offset, unsigned long len); -void memzero_extent_buffer(struct extent_buffer *eb, unsigned long start, - unsigned long len); -int extent_buffer_test_bit(struct extent_buffer *eb, unsigned long start, - unsigned long pos); -void extent_buffer_bitmap_set(struct extent_buffer *eb, unsigned long start, - unsigned long pos, unsigned long len); -void extent_buffer_bitmap_clear(struct extent_buffer *eb, unsigned long start, - unsigned long pos, unsigned long len); -void clear_extent_buffer_dirty(struct extent_buffer *eb); -bool set_extent_buffer_dirty(struct extent_buffer *eb); -void set_extent_buffer_uptodate(struct extent_buffer *eb); -void clear_extent_buffer_uptodate(struct extent_buffer *eb); -int extent_buffer_under_io(struct extent_buffer *eb); -int map_private_extent_buffer(const struct extent_buffer *eb, - unsigned long offset, unsigned long min_len, - char **map, unsigned long *map_start, - unsigned long *map_len); void extent_range_clear_dirty_for_io(struct inode *inode, u64 start, u64 end); void extent_range_redirty_for_io(struct inode *inode, u64 start, u64 end); void extent_clear_unlock_delalloc(struct inode *inode, u64 start, u64 end, @@ -330,8 +190,4 @@ bool find_lock_delalloc_range(struct inode *inode, struct page *locked_page, u64 *start, u64 *end); #endif -struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start); -int __init extent_buffer_init(void); -void __cold extent_buffer_exit(void); #endif From patchwork Tue Sep 17 18:43:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 11149315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7846814DB for ; Tue, 17 Sep 2019 18:44:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 26242206C2 for ; Tue, 17 Sep 2019 18:44:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=toxicpanda-com.20150623.gappssmtp.com header.i=@toxicpanda-com.20150623.gappssmtp.com header.b="GKjb2EnQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729535AbfIQSoI (ORCPT ); Tue, 17 Sep 2019 14:44:08 -0400 Received: from mail-qt1-f196.google.com ([209.85.160.196]:35779 "EHLO mail-qt1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728655AbfIQSoI (ORCPT ); Tue, 17 Sep 2019 14:44:08 -0400 Received: by mail-qt1-f196.google.com with SMTP id m15so5727680qtq.2 for ; Tue, 17 Sep 2019 11:44:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=toxicpanda-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ia2od0WlNoKzPcglkXLc0qZQ+IrnT50pxDcxsYipwZ0=; b=GKjb2EnQwI0bcOLxmhzB1RssPKzvcaQNPtI8ATBZcz7r119uLcZG0im7PVO0jL+9D/ muXCyfiMHrj5JRRCiJLpf5SQwClUBgBwokVXN5G3I2mTPUEStKmnNXa+tmk9GMfBsBwS GwxTnttmFaplT0KTTIVLoNsvdzWxJtIEM5s4Y6LF6agxVmeEAd3soNwOJA9BZVhDMkj3 2TJfqAukbQscW/OtgpYAt4mN7bcoUED+7R6q26ll1fUNF1ltZf7Ve8N60xKMeD3sf0FY 7LGtUX7jaFflUTMLZftWOrE8+JQuaKGi6mDdSANole5FPiBNeY/4Blx1ipBmJQkHtfpb Vo1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ia2od0WlNoKzPcglkXLc0qZQ+IrnT50pxDcxsYipwZ0=; b=VKJQ5HZ/SxwwKXag6jDvX6voP1w5VIQInuxGhRQ3eznCKEfw92c+j6DHainNjFBua8 SguRZF3sJJTTZnJsQQU0n2BCB9uuvagAMqU/mz/D0JNhsf8deoVeE6I0uTAo7+6fe/N0 zrmVAv5+UarZOzcg3RDx0A8f6atkDoeFpDJnJuZMu3QNOZ0YYhY12HdrOV1EqcUL2Wsu hO8OYLwmh5xZm47NpWsmGX2SispR3Im8Ae33u25+pZqx1sFedWAUzWg1e/SeFkSzkuIs Y8SiBuizhrKP09+rziQ/wQhyafHt/ZHIhvuFKu+x1bFfiXuHBG4jVSWnczATSm9embN9 KK2g== X-Gm-Message-State: APjAAAWJ4+aCKiCAByJY8H0E+aHmI7kHldPhp5SbeoSvQ5PjuJ/cVJ2D x+btLlqo7pAc9w6CXATQdFDE41sdaZHNWQ== X-Google-Smtp-Source: APXvYqxCKHInmdiIZHVVnk7xpOSZqL0FNYyDItJdE5SkKe9e9NC2DxRcDFgxihgSh/q+NuLViNKaDw== X-Received: by 2002:a05:6214:2b0:: with SMTP id m16mr4520162qvv.45.1568745844194; Tue, 17 Sep 2019 11:44:04 -0700 (PDT) Received: from localhost ([107.15.81.208]) by smtp.gmail.com with ESMTPSA id u43sm2121499qte.19.2019.09.17.11.44.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Sep 2019 11:44:03 -0700 (PDT) From: Josef Bacik To: linux-btrfs@vger.kernel.org, kernel-team@fb.com Subject: [PATCH 9/9] btrfs: move the extent-buffer code Date: Tue, 17 Sep 2019 14:43:43 -0400 Message-Id: <20190917184344.13155-10-josef@toxicpanda.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190917184344.13155-1-josef@toxicpanda.com> References: <20190917184344.13155-1-josef@toxicpanda.com> MIME-Version: 1.0 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This moves the extent buffer code into its own file. Signed-off-by: Josef Bacik --- fs/btrfs/Makefile | 3 +- fs/btrfs/extent-buffer.c | 1266 ++++++++++++++++++++++++++++++++++++++ fs/btrfs/extent_io.c | 1255 ------------------------------------- 3 files changed, 1268 insertions(+), 1256 deletions(-) create mode 100644 fs/btrfs/extent-buffer.c diff --git a/fs/btrfs/Makefile b/fs/btrfs/Makefile index 7ce7202f5fe8..4d669c1d09fa 100644 --- a/fs/btrfs/Makefile +++ b/fs/btrfs/Makefile @@ -11,7 +11,8 @@ btrfs-y += super.o ctree.o extent-tree.o print-tree.o root-tree.o dir-item.o \ compression.o delayed-ref.o relocation.o delayed-inode.o scrub.o \ reada.o backref.o ulist.o qgroup.o send.o dev-replace.o raid56.o \ uuid-tree.o props.o free-space-tree.o tree-checker.o space-info.o \ - block-rsv.o delalloc-space.o block-group.o extent-io-tree.o + block-rsv.o delalloc-space.o block-group.o extent-io-tree.o \ + extent-buffer.o btrfs-$(CONFIG_BTRFS_FS_POSIX_ACL) += acl.o btrfs-$(CONFIG_BTRFS_FS_CHECK_INTEGRITY) += check-integrity.o diff --git a/fs/btrfs/extent-buffer.c b/fs/btrfs/extent-buffer.c new file mode 100644 index 000000000000..f03e67de34c2 --- /dev/null +++ b/fs/btrfs/extent-buffer.c @@ -0,0 +1,1266 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#include +#include +#include +#include +#include +#include +#include +#include "ctree.h" +#include "extent-buffer.h" + +static struct kmem_cache *extent_buffer_cache; + +#ifdef CONFIG_BTRFS_DEBUG +static LIST_HEAD(buffers); + +static DEFINE_SPINLOCK(leak_lock); + +static inline +void btrfs_leak_debug_add(struct list_head *new, struct list_head *head) +{ + unsigned long flags; + + spin_lock_irqsave(&leak_lock, flags); + list_add(new, head); + spin_unlock_irqrestore(&leak_lock, flags); +} + +static inline +void btrfs_leak_debug_del(struct list_head *entry) +{ + unsigned long flags; + + spin_lock_irqsave(&leak_lock, flags); + list_del(entry); + spin_unlock_irqrestore(&leak_lock, flags); +} + +static inline +void btrfs_extent_buffer_leak_debug_check() +{ + while (!list_empty(&buffers)) { + eb = list_entry(buffers.next, struct extent_buffer, leak_list); + pr_err("BTRFS: buffer leak start %llu len %lu refs %d bflags %lu\n", + eb->start, eb->len, atomic_read(&eb->refs), eb->bflags); + list_del(&eb->leak_list); + kmem_cache_free(extent_buffer_cache, eb); + } +} +#else +#define btrfs_leak_debug_add(new, head) do {} while (0) +#define btrfs_leak_debug_del(entry) do {} while (0) +#define btrfs_extent_buffer_leak_debug_check() do {} while (0) +#endif + +int __init extent_buffer_init(void) +{ + extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer", + sizeof(struct extent_buffer), 0, + SLAB_MEM_SPREAD, NULL); + if (!extent_buffer_cache) + return -ENOMEM; + return 0; +} + +void __cold extent_buffer_exit(void) +{ + btrfs_extent_buffer_leak_debug_check(); + + /* + * Make sure all delayed rcu free are flushed before we + * destroy caches. + */ + rcu_barrier(); + kmem_cache_destroy(extent_buffer_cache); +} + +void wait_on_extent_buffer_writeback(struct extent_buffer *eb) +{ + wait_on_bit_io(&eb->bflags, EXTENT_BUFFER_WRITEBACK, + TASK_UNINTERRUPTIBLE); +} + +static void __free_extent_buffer(struct extent_buffer *eb) +{ + btrfs_leak_debug_del(&eb->leak_list); + kmem_cache_free(extent_buffer_cache, eb); +} + +int extent_buffer_under_io(struct extent_buffer *eb) +{ + return (atomic_read(&eb->io_pages) || + test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) || + test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)); +} + +/* + * Release all pages attached to the extent buffer. + */ +static void btrfs_release_extent_buffer_pages(struct extent_buffer *eb) +{ + int i; + int num_pages; + int mapped = !test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags); + + BUG_ON(extent_buffer_under_io(eb)); + + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++) { + struct page *page = eb->pages[i]; + + if (!page) + continue; + if (mapped) + spin_lock(&page->mapping->private_lock); + /* + * We do this since we'll remove the pages after we've + * removed the eb from the radix tree, so we could race + * and have this page now attached to the new eb. So + * only clear page_private if it's still connected to + * this eb. + */ + if (PagePrivate(page) && + page->private == (unsigned long)eb) { + BUG_ON(test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)); + BUG_ON(PageDirty(page)); + BUG_ON(PageWriteback(page)); + /* + * We need to make sure we haven't be attached + * to a new eb. + */ + ClearPagePrivate(page); + set_page_private(page, 0); + /* One for the page private */ + put_page(page); + } + + if (mapped) + spin_unlock(&page->mapping->private_lock); + + /* One for when we allocated the page */ + put_page(page); + } +} + +/* + * Helper for releasing the extent buffer. + */ +static inline void btrfs_release_extent_buffer(struct extent_buffer *eb) +{ + btrfs_release_extent_buffer_pages(eb); + __free_extent_buffer(eb); +} + +static struct extent_buffer * +__alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start, + unsigned long len) +{ + struct extent_buffer *eb = NULL; + + eb = kmem_cache_zalloc(extent_buffer_cache, GFP_NOFS|__GFP_NOFAIL); + eb->start = start; + eb->len = len; + eb->fs_info = fs_info; + eb->bflags = 0; + rwlock_init(&eb->lock); + atomic_set(&eb->blocking_readers, 0); + eb->blocking_writers = 0; + eb->lock_nested = false; + init_waitqueue_head(&eb->write_lock_wq); + init_waitqueue_head(&eb->read_lock_wq); + + btrfs_leak_debug_add(&eb->leak_list, &buffers); + + spin_lock_init(&eb->refs_lock); + atomic_set(&eb->refs, 1); + atomic_set(&eb->io_pages, 0); + + /* + * Sanity checks, currently the maximum is 64k covered by 16x 4k pages + */ + BUILD_BUG_ON(BTRFS_MAX_METADATA_BLOCKSIZE + > MAX_INLINE_EXTENT_BUFFER_SIZE); + BUG_ON(len > MAX_INLINE_EXTENT_BUFFER_SIZE); + +#ifdef CONFIG_BTRFS_DEBUG + eb->spinning_writers = 0; + atomic_set(&eb->spinning_readers, 0); + atomic_set(&eb->read_locks, 0); + eb->write_locks = 0; +#endif + + return eb; +} + +static void attach_extent_buffer_page(struct extent_buffer *eb, + struct page *page) +{ + if (!PagePrivate(page)) { + SetPagePrivate(page); + get_page(page); + set_page_private(page, (unsigned long)eb); + } else { + WARN_ON(page->private != (unsigned long)eb); + } +} + +struct extent_buffer *btrfs_clone_extent_buffer(struct extent_buffer *src) +{ + int i; + struct page *p; + struct extent_buffer *new; + int num_pages = num_extent_pages(src); + + new = __alloc_extent_buffer(src->fs_info, src->start, src->len); + if (new == NULL) + return NULL; + + for (i = 0; i < num_pages; i++) { + p = alloc_page(GFP_NOFS); + if (!p) { + btrfs_release_extent_buffer(new); + return NULL; + } + attach_extent_buffer_page(new, p); + WARN_ON(PageDirty(p)); + SetPageUptodate(p); + new->pages[i] = p; + copy_page(page_address(p), page_address(src->pages[i])); + } + + set_bit(EXTENT_BUFFER_UPTODATE, &new->bflags); + set_bit(EXTENT_BUFFER_UNMAPPED, &new->bflags); + + return new; +} + +struct extent_buffer *__alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start, unsigned long len) +{ + struct extent_buffer *eb; + int num_pages; + int i; + + eb = __alloc_extent_buffer(fs_info, start, len); + if (!eb) + return NULL; + + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++) { + eb->pages[i] = alloc_page(GFP_NOFS); + if (!eb->pages[i]) + goto err; + } + set_extent_buffer_uptodate(eb); + btrfs_set_header_nritems(eb, 0); + set_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags); + + return eb; +err: + for (; i > 0; i--) + __free_page(eb->pages[i - 1]); + __free_extent_buffer(eb); + return NULL; +} + +struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start) +{ + return __alloc_dummy_extent_buffer(fs_info, start, fs_info->nodesize); +} + +static void check_buffer_tree_ref(struct extent_buffer *eb) +{ + int refs; + /* the ref bit is tricky. We have to make sure it is set + * if we have the buffer dirty. Otherwise the + * code to free a buffer can end up dropping a dirty + * page + * + * Once the ref bit is set, it won't go away while the + * buffer is dirty or in writeback, and it also won't + * go away while we have the reference count on the + * eb bumped. + * + * We can't just set the ref bit without bumping the + * ref on the eb because free_extent_buffer might + * see the ref bit and try to clear it. If this happens + * free_extent_buffer might end up dropping our original + * ref by mistake and freeing the page before we are able + * to add one more ref. + * + * So bump the ref count first, then set the bit. If someone + * beat us to it, drop the ref we added. + */ + refs = atomic_read(&eb->refs); + if (refs >= 2 && test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) + return; + + spin_lock(&eb->refs_lock); + if (!test_and_set_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) + atomic_inc(&eb->refs); + spin_unlock(&eb->refs_lock); +} + +static void mark_extent_buffer_accessed(struct extent_buffer *eb, + struct page *accessed) +{ + int num_pages, i; + + check_buffer_tree_ref(eb); + + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++) { + struct page *p = eb->pages[i]; + + if (p != accessed) + mark_page_accessed(p); + } +} + +struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start) +{ + struct extent_buffer *eb; + + rcu_read_lock(); + eb = radix_tree_lookup(&fs_info->buffer_radix, + start >> PAGE_SHIFT); + if (eb && atomic_inc_not_zero(&eb->refs)) { + rcu_read_unlock(); + /* + * Lock our eb's refs_lock to avoid races with + * free_extent_buffer. When we get our eb it might be flagged + * with EXTENT_BUFFER_STALE and another task running + * free_extent_buffer might have seen that flag set, + * eb->refs == 2, that the buffer isn't under IO (dirty and + * writeback flags not set) and it's still in the tree (flag + * EXTENT_BUFFER_TREE_REF set), therefore being in the process + * of decrementing the extent buffer's reference count twice. + * So here we could race and increment the eb's reference count, + * clear its stale flag, mark it as dirty and drop our reference + * before the other task finishes executing free_extent_buffer, + * which would later result in an attempt to free an extent + * buffer that is dirty. + */ + if (test_bit(EXTENT_BUFFER_STALE, &eb->bflags)) { + spin_lock(&eb->refs_lock); + spin_unlock(&eb->refs_lock); + } + mark_extent_buffer_accessed(eb, NULL); + return eb; + } + rcu_read_unlock(); + + return NULL; +} + +#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS +struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start) +{ + struct extent_buffer *eb, *exists = NULL; + int ret; + + eb = find_extent_buffer(fs_info, start); + if (eb) + return eb; + eb = alloc_dummy_extent_buffer(fs_info, start); + if (!eb) + return NULL; + eb->fs_info = fs_info; +again: + ret = radix_tree_preload(GFP_NOFS); + if (ret) + goto free_eb; + spin_lock(&fs_info->buffer_lock); + ret = radix_tree_insert(&fs_info->buffer_radix, + start >> PAGE_SHIFT, eb); + spin_unlock(&fs_info->buffer_lock); + radix_tree_preload_end(); + if (ret == -EEXIST) { + exists = find_extent_buffer(fs_info, start); + if (exists) + goto free_eb; + else + goto again; + } + check_buffer_tree_ref(eb); + set_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags); + + return eb; +free_eb: + btrfs_release_extent_buffer(eb); + return exists; +} +#endif + +struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info, + u64 start) +{ + unsigned long len = fs_info->nodesize; + int num_pages; + int i; + unsigned long index = start >> PAGE_SHIFT; + struct extent_buffer *eb; + struct extent_buffer *exists = NULL; + struct page *p; + struct address_space *mapping = fs_info->btree_inode->i_mapping; + int uptodate = 1; + int ret; + + if (!IS_ALIGNED(start, fs_info->sectorsize)) { + btrfs_err(fs_info, "bad tree block start %llu", start); + return ERR_PTR(-EINVAL); + } + + eb = find_extent_buffer(fs_info, start); + if (eb) + return eb; + + eb = __alloc_extent_buffer(fs_info, start, len); + if (!eb) + return ERR_PTR(-ENOMEM); + + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++, index++) { + p = find_or_create_page(mapping, index, GFP_NOFS|__GFP_NOFAIL); + if (!p) { + exists = ERR_PTR(-ENOMEM); + goto free_eb; + } + + spin_lock(&mapping->private_lock); + if (PagePrivate(p)) { + /* + * We could have already allocated an eb for this page + * and attached one so lets see if we can get a ref on + * the existing eb, and if we can we know it's good and + * we can just return that one, else we know we can just + * overwrite page->private. + */ + exists = (struct extent_buffer *)p->private; + if (atomic_inc_not_zero(&exists->refs)) { + spin_unlock(&mapping->private_lock); + unlock_page(p); + put_page(p); + mark_extent_buffer_accessed(exists, p); + goto free_eb; + } + exists = NULL; + + /* + * Do this so attach doesn't complain and we need to + * drop the ref the old guy had. + */ + ClearPagePrivate(p); + WARN_ON(PageDirty(p)); + put_page(p); + } + attach_extent_buffer_page(eb, p); + spin_unlock(&mapping->private_lock); + WARN_ON(PageDirty(p)); + eb->pages[i] = p; + if (!PageUptodate(p)) + uptodate = 0; + + /* + * We can't unlock the pages just yet since the extent buffer + * hasn't been properly inserted in the radix tree, this + * opens a race with btree_releasepage which can free a page + * while we are still filling in all pages for the buffer and + * we could crash. + */ + } + if (uptodate) + set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); +again: + ret = radix_tree_preload(GFP_NOFS); + if (ret) { + exists = ERR_PTR(ret); + goto free_eb; + } + + spin_lock(&fs_info->buffer_lock); + ret = radix_tree_insert(&fs_info->buffer_radix, + start >> PAGE_SHIFT, eb); + spin_unlock(&fs_info->buffer_lock); + radix_tree_preload_end(); + if (ret == -EEXIST) { + exists = find_extent_buffer(fs_info, start); + if (exists) + goto free_eb; + else + goto again; + } + /* add one reference for the tree */ + check_buffer_tree_ref(eb); + set_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags); + + /* + * Now it's safe to unlock the pages because any calls to + * btree_releasepage will correctly detect that a page belongs to a + * live buffer and won't free them prematurely. + */ + for (i = 0; i < num_pages; i++) + unlock_page(eb->pages[i]); + return eb; + +free_eb: + WARN_ON(!atomic_dec_and_test(&eb->refs)); + for (i = 0; i < num_pages; i++) { + if (eb->pages[i]) + unlock_page(eb->pages[i]); + } + + btrfs_release_extent_buffer(eb); + return exists; +} + +static inline void btrfs_release_extent_buffer_rcu(struct rcu_head *head) +{ + struct extent_buffer *eb = + container_of(head, struct extent_buffer, rcu_head); + + __free_extent_buffer(eb); +} + +static int release_extent_buffer(struct extent_buffer *eb) +{ + lockdep_assert_held(&eb->refs_lock); + + WARN_ON(atomic_read(&eb->refs) == 0); + if (atomic_dec_and_test(&eb->refs)) { + if (test_and_clear_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags)) { + struct btrfs_fs_info *fs_info = eb->fs_info; + + spin_unlock(&eb->refs_lock); + + spin_lock(&fs_info->buffer_lock); + radix_tree_delete(&fs_info->buffer_radix, + eb->start >> PAGE_SHIFT); + spin_unlock(&fs_info->buffer_lock); + } else { + spin_unlock(&eb->refs_lock); + } + + /* Should be safe to release our pages at this point */ + btrfs_release_extent_buffer_pages(eb); +#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS + if (unlikely(test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags))) { + __free_extent_buffer(eb); + return 1; + } +#endif + call_rcu(&eb->rcu_head, btrfs_release_extent_buffer_rcu); + return 1; + } + spin_unlock(&eb->refs_lock); + + return 0; +} + +void free_extent_buffer(struct extent_buffer *eb) +{ + int refs; + int old; + if (!eb) + return; + + while (1) { + refs = atomic_read(&eb->refs); + if ((!test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags) && refs <= 3) + || (test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags) && + refs == 1)) + break; + old = atomic_cmpxchg(&eb->refs, refs, refs - 1); + if (old == refs) + return; + } + + spin_lock(&eb->refs_lock); + if (atomic_read(&eb->refs) == 2 && + test_bit(EXTENT_BUFFER_STALE, &eb->bflags) && + !extent_buffer_under_io(eb) && + test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) + atomic_dec(&eb->refs); + + /* + * I know this is terrible, but it's temporary until we stop tracking + * the uptodate bits and such for the extent buffers. + */ + release_extent_buffer(eb); +} + +void free_extent_buffer_stale(struct extent_buffer *eb) +{ + if (!eb) + return; + + spin_lock(&eb->refs_lock); + set_bit(EXTENT_BUFFER_STALE, &eb->bflags); + + if (atomic_read(&eb->refs) == 2 && !extent_buffer_under_io(eb) && + test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) + atomic_dec(&eb->refs); + release_extent_buffer(eb); +} + +void clear_extent_buffer_dirty(struct extent_buffer *eb) +{ + int i; + int num_pages; + struct page *page; + + num_pages = num_extent_pages(eb); + + for (i = 0; i < num_pages; i++) { + page = eb->pages[i]; + if (!PageDirty(page)) + continue; + + lock_page(page); + WARN_ON(!PagePrivate(page)); + + clear_page_dirty_for_io(page); + xa_lock_irq(&page->mapping->i_pages); + if (!PageDirty(page)) + __xa_clear_mark(&page->mapping->i_pages, + page_index(page), PAGECACHE_TAG_DIRTY); + xa_unlock_irq(&page->mapping->i_pages); + ClearPageError(page); + unlock_page(page); + } + WARN_ON(atomic_read(&eb->refs) == 0); +} + +bool set_extent_buffer_dirty(struct extent_buffer *eb) +{ + int i; + int num_pages; + bool was_dirty; + + check_buffer_tree_ref(eb); + + was_dirty = test_and_set_bit(EXTENT_BUFFER_DIRTY, &eb->bflags); + + num_pages = num_extent_pages(eb); + WARN_ON(atomic_read(&eb->refs) == 0); + WARN_ON(!test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)); + + if (!was_dirty) + for (i = 0; i < num_pages; i++) + set_page_dirty(eb->pages[i]); + +#ifdef CONFIG_BTRFS_DEBUG + for (i = 0; i < num_pages; i++) + ASSERT(PageDirty(eb->pages[i])); +#endif + + return was_dirty; +} + +void clear_extent_buffer_uptodate(struct extent_buffer *eb) +{ + int i; + struct page *page; + int num_pages; + + clear_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++) { + page = eb->pages[i]; + if (page) + ClearPageUptodate(page); + } +} + +void set_extent_buffer_uptodate(struct extent_buffer *eb) +{ + int i; + struct page *page; + int num_pages; + + set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); + num_pages = num_extent_pages(eb); + for (i = 0; i < num_pages; i++) { + page = eb->pages[i]; + SetPageUptodate(page); + } +} + +void read_extent_buffer(const struct extent_buffer *eb, void *dstv, + unsigned long start, unsigned long len) +{ + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + char *dst = (char *)dstv; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + + if (start + len > eb->len) { + WARN(1, KERN_ERR "btrfs bad mapping eb start %llu len %lu, wanted %lu %lu\n", + eb->start, eb->len, start, len); + memset(dst, 0, len); + return; + } + + offset = offset_in_page(start_offset + start); + + while (len > 0) { + page = eb->pages[i]; + + cur = min(len, (PAGE_SIZE - offset)); + kaddr = page_address(page); + memcpy(dst, kaddr + offset, cur); + + dst += cur; + len -= cur; + offset = 0; + i++; + } +} + +int read_extent_buffer_to_user(const struct extent_buffer *eb, + void __user *dstv, + unsigned long start, unsigned long len) +{ + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + char __user *dst = (char __user *)dstv; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + int ret = 0; + + WARN_ON(start > eb->len); + WARN_ON(start + len > eb->start + eb->len); + + offset = offset_in_page(start_offset + start); + + while (len > 0) { + page = eb->pages[i]; + + cur = min(len, (PAGE_SIZE - offset)); + kaddr = page_address(page); + if (copy_to_user(dst, kaddr + offset, cur)) { + ret = -EFAULT; + break; + } + + dst += cur; + len -= cur; + offset = 0; + i++; + } + + return ret; +} + +/* + * return 0 if the item is found within a page. + * return 1 if the item spans two pages. + * return -EINVAL otherwise. + */ +int map_private_extent_buffer(const struct extent_buffer *eb, + unsigned long start, unsigned long min_len, + char **map, unsigned long *map_start, + unsigned long *map_len) +{ + size_t offset; + char *kaddr; + struct page *p; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + unsigned long end_i = (start_offset + start + min_len - 1) >> + PAGE_SHIFT; + + if (start + min_len > eb->len) { + WARN(1, KERN_ERR "btrfs bad mapping eb start %llu len %lu, wanted %lu %lu\n", + eb->start, eb->len, start, min_len); + return -EINVAL; + } + + if (i != end_i) + return 1; + + if (i == 0) { + offset = start_offset; + *map_start = 0; + } else { + offset = 0; + *map_start = ((u64)i << PAGE_SHIFT) - start_offset; + } + + p = eb->pages[i]; + kaddr = page_address(p); + *map = kaddr + offset; + *map_len = PAGE_SIZE - offset; + return 0; +} + +int memcmp_extent_buffer(const struct extent_buffer *eb, const void *ptrv, + unsigned long start, unsigned long len) +{ + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + char *ptr = (char *)ptrv; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + int ret = 0; + + WARN_ON(start > eb->len); + WARN_ON(start + len > eb->start + eb->len); + + offset = offset_in_page(start_offset + start); + + while (len > 0) { + page = eb->pages[i]; + + cur = min(len, (PAGE_SIZE - offset)); + + kaddr = page_address(page); + ret = memcmp(ptr, kaddr + offset, cur); + if (ret) + break; + + ptr += cur; + len -= cur; + offset = 0; + i++; + } + return ret; +} + +void write_extent_buffer_chunk_tree_uuid(struct extent_buffer *eb, + const void *srcv) +{ + char *kaddr; + + WARN_ON(!PageUptodate(eb->pages[0])); + kaddr = page_address(eb->pages[0]); + memcpy(kaddr + offsetof(struct btrfs_header, chunk_tree_uuid), srcv, + BTRFS_FSID_SIZE); +} + +void write_extent_buffer_fsid(struct extent_buffer *eb, const void *srcv) +{ + char *kaddr; + + WARN_ON(!PageUptodate(eb->pages[0])); + kaddr = page_address(eb->pages[0]); + memcpy(kaddr + offsetof(struct btrfs_header, fsid), srcv, + BTRFS_FSID_SIZE); +} + +void write_extent_buffer(struct extent_buffer *eb, const void *srcv, + unsigned long start, unsigned long len) +{ + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + char *src = (char *)srcv; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + + WARN_ON(start > eb->len); + WARN_ON(start + len > eb->start + eb->len); + + offset = offset_in_page(start_offset + start); + + while (len > 0) { + page = eb->pages[i]; + WARN_ON(!PageUptodate(page)); + + cur = min(len, PAGE_SIZE - offset); + kaddr = page_address(page); + memcpy(kaddr + offset, src, cur); + + src += cur; + len -= cur; + offset = 0; + i++; + } +} + +void memzero_extent_buffer(struct extent_buffer *eb, unsigned long start, + unsigned long len) +{ + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + size_t start_offset = offset_in_page(eb->start); + unsigned long i = (start_offset + start) >> PAGE_SHIFT; + + WARN_ON(start > eb->len); + WARN_ON(start + len > eb->start + eb->len); + + offset = offset_in_page(start_offset + start); + + while (len > 0) { + page = eb->pages[i]; + WARN_ON(!PageUptodate(page)); + + cur = min(len, PAGE_SIZE - offset); + kaddr = page_address(page); + memset(kaddr + offset, 0, cur); + + len -= cur; + offset = 0; + i++; + } +} + +void copy_extent_buffer_full(struct extent_buffer *dst, + struct extent_buffer *src) +{ + int i; + int num_pages; + + ASSERT(dst->len == src->len); + + num_pages = num_extent_pages(dst); + for (i = 0; i < num_pages; i++) + copy_page(page_address(dst->pages[i]), + page_address(src->pages[i])); +} + +void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src, + unsigned long dst_offset, unsigned long src_offset, + unsigned long len) +{ + u64 dst_len = dst->len; + size_t cur; + size_t offset; + struct page *page; + char *kaddr; + size_t start_offset = offset_in_page(dst->start); + unsigned long i = (start_offset + dst_offset) >> PAGE_SHIFT; + + WARN_ON(src->len != dst_len); + + offset = offset_in_page(start_offset + dst_offset); + + while (len > 0) { + page = dst->pages[i]; + WARN_ON(!PageUptodate(page)); + + cur = min(len, (unsigned long)(PAGE_SIZE - offset)); + + kaddr = page_address(page); + read_extent_buffer(src, kaddr + offset, src_offset, cur); + + src_offset += cur; + len -= cur; + offset = 0; + i++; + } +} + +/* + * eb_bitmap_offset() - calculate the page and offset of the byte containing the + * given bit number + * @eb: the extent buffer + * @start: offset of the bitmap item in the extent buffer + * @nr: bit number + * @page_index: return index of the page in the extent buffer that contains the + * given bit number + * @page_offset: return offset into the page given by page_index + * + * This helper hides the ugliness of finding the byte in an extent buffer which + * contains a given bit. + */ +static inline void eb_bitmap_offset(struct extent_buffer *eb, + unsigned long start, unsigned long nr, + unsigned long *page_index, + size_t *page_offset) +{ + size_t start_offset = offset_in_page(eb->start); + size_t byte_offset = BIT_BYTE(nr); + size_t offset; + + /* + * The byte we want is the offset of the extent buffer + the offset of + * the bitmap item in the extent buffer + the offset of the byte in the + * bitmap item. + */ + offset = start_offset + start + byte_offset; + + *page_index = offset >> PAGE_SHIFT; + *page_offset = offset_in_page(offset); +} + +/** + * extent_buffer_test_bit - determine whether a bit in a bitmap item is set + * @eb: the extent buffer + * @start: offset of the bitmap item in the extent buffer + * @nr: bit number to test + */ +int extent_buffer_test_bit(struct extent_buffer *eb, unsigned long start, + unsigned long nr) +{ + u8 *kaddr; + struct page *page; + unsigned long i; + size_t offset; + + eb_bitmap_offset(eb, start, nr, &i, &offset); + page = eb->pages[i]; + WARN_ON(!PageUptodate(page)); + kaddr = page_address(page); + return 1U & (kaddr[offset] >> (nr & (BITS_PER_BYTE - 1))); +} + +/** + * extent_buffer_bitmap_set - set an area of a bitmap + * @eb: the extent buffer + * @start: offset of the bitmap item in the extent buffer + * @pos: bit number of the first bit + * @len: number of bits to set + */ +void extent_buffer_bitmap_set(struct extent_buffer *eb, unsigned long start, + unsigned long pos, unsigned long len) +{ + u8 *kaddr; + struct page *page; + unsigned long i; + size_t offset; + const unsigned int size = pos + len; + int bits_to_set = BITS_PER_BYTE - (pos % BITS_PER_BYTE); + u8 mask_to_set = BITMAP_FIRST_BYTE_MASK(pos); + + eb_bitmap_offset(eb, start, pos, &i, &offset); + page = eb->pages[i]; + WARN_ON(!PageUptodate(page)); + kaddr = page_address(page); + + while (len >= bits_to_set) { + kaddr[offset] |= mask_to_set; + len -= bits_to_set; + bits_to_set = BITS_PER_BYTE; + mask_to_set = ~0; + if (++offset >= PAGE_SIZE && len > 0) { + offset = 0; + page = eb->pages[++i]; + WARN_ON(!PageUptodate(page)); + kaddr = page_address(page); + } + } + if (len) { + mask_to_set &= BITMAP_LAST_BYTE_MASK(size); + kaddr[offset] |= mask_to_set; + } +} + + +/** + * extent_buffer_bitmap_clear - clear an area of a bitmap + * @eb: the extent buffer + * @start: offset of the bitmap item in the extent buffer + * @pos: bit number of the first bit + * @len: number of bits to clear + */ +void extent_buffer_bitmap_clear(struct extent_buffer *eb, unsigned long start, + unsigned long pos, unsigned long len) +{ + u8 *kaddr; + struct page *page; + unsigned long i; + size_t offset; + const unsigned int size = pos + len; + int bits_to_clear = BITS_PER_BYTE - (pos % BITS_PER_BYTE); + u8 mask_to_clear = BITMAP_FIRST_BYTE_MASK(pos); + + eb_bitmap_offset(eb, start, pos, &i, &offset); + page = eb->pages[i]; + WARN_ON(!PageUptodate(page)); + kaddr = page_address(page); + + while (len >= bits_to_clear) { + kaddr[offset] &= ~mask_to_clear; + len -= bits_to_clear; + bits_to_clear = BITS_PER_BYTE; + mask_to_clear = ~0; + if (++offset >= PAGE_SIZE && len > 0) { + offset = 0; + page = eb->pages[++i]; + WARN_ON(!PageUptodate(page)); + kaddr = page_address(page); + } + } + if (len) { + mask_to_clear &= BITMAP_LAST_BYTE_MASK(size); + kaddr[offset] &= ~mask_to_clear; + } +} + +static inline bool areas_overlap(unsigned long src, unsigned long dst, unsigned long len) +{ + unsigned long distance = (src > dst) ? src - dst : dst - src; + return distance < len; +} + +static void copy_pages(struct page *dst_page, struct page *src_page, + unsigned long dst_off, unsigned long src_off, + unsigned long len) +{ + char *dst_kaddr = page_address(dst_page); + char *src_kaddr; + int must_memmove = 0; + + if (dst_page != src_page) { + src_kaddr = page_address(src_page); + } else { + src_kaddr = dst_kaddr; + if (areas_overlap(src_off, dst_off, len)) + must_memmove = 1; + } + + if (must_memmove) + memmove(dst_kaddr + dst_off, src_kaddr + src_off, len); + else + memcpy(dst_kaddr + dst_off, src_kaddr + src_off, len); +} + +void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, + unsigned long src_offset, unsigned long len) +{ + struct btrfs_fs_info *fs_info = dst->fs_info; + size_t cur; + size_t dst_off_in_page; + size_t src_off_in_page; + size_t start_offset = offset_in_page(dst->start); + unsigned long dst_i; + unsigned long src_i; + + if (src_offset + len > dst->len) { + btrfs_err(fs_info, + "memmove bogus src_offset %lu move len %lu dst len %lu", + src_offset, len, dst->len); + BUG(); + } + if (dst_offset + len > dst->len) { + btrfs_err(fs_info, + "memmove bogus dst_offset %lu move len %lu dst len %lu", + dst_offset, len, dst->len); + BUG(); + } + + while (len > 0) { + dst_off_in_page = offset_in_page(start_offset + dst_offset); + src_off_in_page = offset_in_page(start_offset + src_offset); + + dst_i = (start_offset + dst_offset) >> PAGE_SHIFT; + src_i = (start_offset + src_offset) >> PAGE_SHIFT; + + cur = min(len, (unsigned long)(PAGE_SIZE - + src_off_in_page)); + cur = min_t(unsigned long, cur, + (unsigned long)(PAGE_SIZE - dst_off_in_page)); + + copy_pages(dst->pages[dst_i], dst->pages[src_i], + dst_off_in_page, src_off_in_page, cur); + + src_offset += cur; + dst_offset += cur; + len -= cur; + } +} + +void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, + unsigned long src_offset, unsigned long len) +{ + struct btrfs_fs_info *fs_info = dst->fs_info; + size_t cur; + size_t dst_off_in_page; + size_t src_off_in_page; + unsigned long dst_end = dst_offset + len - 1; + unsigned long src_end = src_offset + len - 1; + size_t start_offset = offset_in_page(dst->start); + unsigned long dst_i; + unsigned long src_i; + + if (src_offset + len > dst->len) { + btrfs_err(fs_info, + "memmove bogus src_offset %lu move len %lu len %lu", + src_offset, len, dst->len); + BUG(); + } + if (dst_offset + len > dst->len) { + btrfs_err(fs_info, + "memmove bogus dst_offset %lu move len %lu len %lu", + dst_offset, len, dst->len); + BUG(); + } + if (dst_offset < src_offset) { + memcpy_extent_buffer(dst, dst_offset, src_offset, len); + return; + } + while (len > 0) { + dst_i = (start_offset + dst_end) >> PAGE_SHIFT; + src_i = (start_offset + src_end) >> PAGE_SHIFT; + + dst_off_in_page = offset_in_page(start_offset + dst_end); + src_off_in_page = offset_in_page(start_offset + src_end); + + cur = min_t(unsigned long, len, src_off_in_page + 1); + cur = min(cur, dst_off_in_page + 1); + copy_pages(dst->pages[dst_i], dst->pages[src_i], + dst_off_in_page - cur + 1, + src_off_in_page - cur + 1, cur); + + dst_end -= cur; + src_end -= cur; + len -= cur; + } +} + +int try_release_extent_buffer(struct page *page) +{ + struct extent_buffer *eb; + + /* + * We need to make sure nobody is attaching this page to an eb right + * now. + */ + spin_lock(&page->mapping->private_lock); + if (!PagePrivate(page)) { + spin_unlock(&page->mapping->private_lock); + return 1; + } + + eb = (struct extent_buffer *)page->private; + BUG_ON(!eb); + + /* + * This is a little awful but should be ok, we need to make sure that + * the eb doesn't disappear out from under us while we're looking at + * this page. + */ + spin_lock(&eb->refs_lock); + if (atomic_read(&eb->refs) != 1 || extent_buffer_under_io(eb)) { + spin_unlock(&eb->refs_lock); + spin_unlock(&page->mapping->private_lock); + return 0; + } + spin_unlock(&page->mapping->private_lock); + + /* + * If tree ref isn't set then we know the ref on this eb is a real ref, + * so just return, this page will likely be freed soon anyway. + */ + if (!test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) { + spin_unlock(&eb->refs_lock); + return 0; + } + + return release_extent_buffer(eb); +} diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index db85c79e2f90..a6a659d91192 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -25,51 +25,8 @@ #include "backref.h" #include "disk-io.h" -static struct kmem_cache *extent_buffer_cache; static struct bio_set btrfs_bioset; -#ifdef CONFIG_BTRFS_DEBUG -static LIST_HEAD(buffers); - -static DEFINE_SPINLOCK(leak_lock); - -static inline -void btrfs_leak_debug_add(struct list_head *new, struct list_head *head) -{ - unsigned long flags; - - spin_lock_irqsave(&leak_lock, flags); - list_add(new, head); - spin_unlock_irqrestore(&leak_lock, flags); -} - -static inline -void btrfs_leak_debug_del(struct list_head *entry) -{ - unsigned long flags; - - spin_lock_irqsave(&leak_lock, flags); - list_del(entry); - spin_unlock_irqrestore(&leak_lock, flags); -} - -static inline -void btrfs_extent_buffer_leak_debug_check() -{ - while (!list_empty(&buffers)) { - eb = list_entry(buffers.next, struct extent_buffer, leak_list); - pr_err("BTRFS: buffer leak start %llu len %lu refs %d bflags %lu\n", - eb->start, eb->len, atomic_read(&eb->refs), eb->bflags); - list_del(&eb->leak_list); - kmem_cache_free(extent_buffer_cache, eb); - } -} -#else -#define btrfs_leak_debug_add(new, head) do {} while (0) -#define btrfs_leak_debug_del(entry) do {} while (0) -#define btrfs_extent_buffer_leak_debug_check() do {} while (0) -#endif - struct extent_page_data { struct bio *bio; struct extent_io_tree *tree; @@ -148,28 +105,6 @@ int __init extent_io_init(void) return 0; } -int __init extent_buffer_init(void) -{ - extent_buffer_cache = kmem_cache_create("btrfs_extent_buffer", - sizeof(struct extent_buffer), 0, - SLAB_MEM_SPREAD, NULL); - if (!extent_buffer_cache) - return -ENOMEM; - return 0; -} - -void __cold extent_buffer_exit(void) -{ - btrfs_extent_buffer_leak_debug_check(); - - /* - * Make sure all delayed rcu free are flushed before we - * destroy caches. - */ - rcu_barrier(); - kmem_cache_destroy(extent_buffer_cache); -} - void __cold extent_io_exit(void) { bioset_exit(&btrfs_bioset); @@ -1089,18 +1024,6 @@ static int submit_extent_page(unsigned int opf, struct extent_io_tree *tree, return ret; } -static void attach_extent_buffer_page(struct extent_buffer *eb, - struct page *page) -{ - if (!PagePrivate(page)) { - SetPagePrivate(page); - get_page(page); - set_page_private(page, (unsigned long)eb); - } else { - WARN_ON(page->private != (unsigned long)eb); - } -} - void set_page_extent_mapped(struct page *page) { if (!PagePrivate(page)) { @@ -1718,12 +1641,6 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc, return 0; } -void wait_on_extent_buffer_writeback(struct extent_buffer *eb) -{ - wait_on_bit_io(&eb->bflags, EXTENT_BUFFER_WRITEBACK, - TASK_UNINTERRUPTIBLE); -} - /* * Lock eb pages and flush the bio if we can't the locks * @@ -2867,603 +2784,6 @@ int extent_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, return ret; } -static void __free_extent_buffer(struct extent_buffer *eb) -{ - btrfs_leak_debug_del(&eb->leak_list); - kmem_cache_free(extent_buffer_cache, eb); -} - -int extent_buffer_under_io(struct extent_buffer *eb) -{ - return (atomic_read(&eb->io_pages) || - test_bit(EXTENT_BUFFER_WRITEBACK, &eb->bflags) || - test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)); -} - -/* - * Release all pages attached to the extent buffer. - */ -static void btrfs_release_extent_buffer_pages(struct extent_buffer *eb) -{ - int i; - int num_pages; - int mapped = !test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags); - - BUG_ON(extent_buffer_under_io(eb)); - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - struct page *page = eb->pages[i]; - - if (!page) - continue; - if (mapped) - spin_lock(&page->mapping->private_lock); - /* - * We do this since we'll remove the pages after we've - * removed the eb from the radix tree, so we could race - * and have this page now attached to the new eb. So - * only clear page_private if it's still connected to - * this eb. - */ - if (PagePrivate(page) && - page->private == (unsigned long)eb) { - BUG_ON(test_bit(EXTENT_BUFFER_DIRTY, &eb->bflags)); - BUG_ON(PageDirty(page)); - BUG_ON(PageWriteback(page)); - /* - * We need to make sure we haven't be attached - * to a new eb. - */ - ClearPagePrivate(page); - set_page_private(page, 0); - /* One for the page private */ - put_page(page); - } - - if (mapped) - spin_unlock(&page->mapping->private_lock); - - /* One for when we allocated the page */ - put_page(page); - } -} - -/* - * Helper for releasing the extent buffer. - */ -static inline void btrfs_release_extent_buffer(struct extent_buffer *eb) -{ - btrfs_release_extent_buffer_pages(eb); - __free_extent_buffer(eb); -} - -static struct extent_buffer * -__alloc_extent_buffer(struct btrfs_fs_info *fs_info, u64 start, - unsigned long len) -{ - struct extent_buffer *eb = NULL; - - eb = kmem_cache_zalloc(extent_buffer_cache, GFP_NOFS|__GFP_NOFAIL); - eb->start = start; - eb->len = len; - eb->fs_info = fs_info; - eb->bflags = 0; - rwlock_init(&eb->lock); - atomic_set(&eb->blocking_readers, 0); - eb->blocking_writers = 0; - eb->lock_nested = false; - init_waitqueue_head(&eb->write_lock_wq); - init_waitqueue_head(&eb->read_lock_wq); - - btrfs_leak_debug_add(&eb->leak_list, &buffers); - - spin_lock_init(&eb->refs_lock); - atomic_set(&eb->refs, 1); - atomic_set(&eb->io_pages, 0); - - /* - * Sanity checks, currently the maximum is 64k covered by 16x 4k pages - */ - BUILD_BUG_ON(BTRFS_MAX_METADATA_BLOCKSIZE - > MAX_INLINE_EXTENT_BUFFER_SIZE); - BUG_ON(len > MAX_INLINE_EXTENT_BUFFER_SIZE); - -#ifdef CONFIG_BTRFS_DEBUG - eb->spinning_writers = 0; - atomic_set(&eb->spinning_readers, 0); - atomic_set(&eb->read_locks, 0); - eb->write_locks = 0; -#endif - - return eb; -} - -struct extent_buffer *btrfs_clone_extent_buffer(struct extent_buffer *src) -{ - int i; - struct page *p; - struct extent_buffer *new; - int num_pages = num_extent_pages(src); - - new = __alloc_extent_buffer(src->fs_info, src->start, src->len); - if (new == NULL) - return NULL; - - for (i = 0; i < num_pages; i++) { - p = alloc_page(GFP_NOFS); - if (!p) { - btrfs_release_extent_buffer(new); - return NULL; - } - attach_extent_buffer_page(new, p); - WARN_ON(PageDirty(p)); - SetPageUptodate(p); - new->pages[i] = p; - copy_page(page_address(p), page_address(src->pages[i])); - } - - set_bit(EXTENT_BUFFER_UPTODATE, &new->bflags); - set_bit(EXTENT_BUFFER_UNMAPPED, &new->bflags); - - return new; -} - -struct extent_buffer *__alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start, unsigned long len) -{ - struct extent_buffer *eb; - int num_pages; - int i; - - eb = __alloc_extent_buffer(fs_info, start, len); - if (!eb) - return NULL; - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - eb->pages[i] = alloc_page(GFP_NOFS); - if (!eb->pages[i]) - goto err; - } - set_extent_buffer_uptodate(eb); - btrfs_set_header_nritems(eb, 0); - set_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags); - - return eb; -err: - for (; i > 0; i--) - __free_page(eb->pages[i - 1]); - __free_extent_buffer(eb); - return NULL; -} - -struct extent_buffer *alloc_dummy_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start) -{ - return __alloc_dummy_extent_buffer(fs_info, start, fs_info->nodesize); -} - -static void check_buffer_tree_ref(struct extent_buffer *eb) -{ - int refs; - /* the ref bit is tricky. We have to make sure it is set - * if we have the buffer dirty. Otherwise the - * code to free a buffer can end up dropping a dirty - * page - * - * Once the ref bit is set, it won't go away while the - * buffer is dirty or in writeback, and it also won't - * go away while we have the reference count on the - * eb bumped. - * - * We can't just set the ref bit without bumping the - * ref on the eb because free_extent_buffer might - * see the ref bit and try to clear it. If this happens - * free_extent_buffer might end up dropping our original - * ref by mistake and freeing the page before we are able - * to add one more ref. - * - * So bump the ref count first, then set the bit. If someone - * beat us to it, drop the ref we added. - */ - refs = atomic_read(&eb->refs); - if (refs >= 2 && test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) - return; - - spin_lock(&eb->refs_lock); - if (!test_and_set_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) - atomic_inc(&eb->refs); - spin_unlock(&eb->refs_lock); -} - -static void mark_extent_buffer_accessed(struct extent_buffer *eb, - struct page *accessed) -{ - int num_pages, i; - - check_buffer_tree_ref(eb); - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - struct page *p = eb->pages[i]; - - if (p != accessed) - mark_page_accessed(p); - } -} - -struct extent_buffer *find_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start) -{ - struct extent_buffer *eb; - - rcu_read_lock(); - eb = radix_tree_lookup(&fs_info->buffer_radix, - start >> PAGE_SHIFT); - if (eb && atomic_inc_not_zero(&eb->refs)) { - rcu_read_unlock(); - /* - * Lock our eb's refs_lock to avoid races with - * free_extent_buffer. When we get our eb it might be flagged - * with EXTENT_BUFFER_STALE and another task running - * free_extent_buffer might have seen that flag set, - * eb->refs == 2, that the buffer isn't under IO (dirty and - * writeback flags not set) and it's still in the tree (flag - * EXTENT_BUFFER_TREE_REF set), therefore being in the process - * of decrementing the extent buffer's reference count twice. - * So here we could race and increment the eb's reference count, - * clear its stale flag, mark it as dirty and drop our reference - * before the other task finishes executing free_extent_buffer, - * which would later result in an attempt to free an extent - * buffer that is dirty. - */ - if (test_bit(EXTENT_BUFFER_STALE, &eb->bflags)) { - spin_lock(&eb->refs_lock); - spin_unlock(&eb->refs_lock); - } - mark_extent_buffer_accessed(eb, NULL); - return eb; - } - rcu_read_unlock(); - - return NULL; -} - -#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS -struct extent_buffer *alloc_test_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start) -{ - struct extent_buffer *eb, *exists = NULL; - int ret; - - eb = find_extent_buffer(fs_info, start); - if (eb) - return eb; - eb = alloc_dummy_extent_buffer(fs_info, start); - if (!eb) - return NULL; - eb->fs_info = fs_info; -again: - ret = radix_tree_preload(GFP_NOFS); - if (ret) - goto free_eb; - spin_lock(&fs_info->buffer_lock); - ret = radix_tree_insert(&fs_info->buffer_radix, - start >> PAGE_SHIFT, eb); - spin_unlock(&fs_info->buffer_lock); - radix_tree_preload_end(); - if (ret == -EEXIST) { - exists = find_extent_buffer(fs_info, start); - if (exists) - goto free_eb; - else - goto again; - } - check_buffer_tree_ref(eb); - set_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags); - - return eb; -free_eb: - btrfs_release_extent_buffer(eb); - return exists; -} -#endif - -struct extent_buffer *alloc_extent_buffer(struct btrfs_fs_info *fs_info, - u64 start) -{ - unsigned long len = fs_info->nodesize; - int num_pages; - int i; - unsigned long index = start >> PAGE_SHIFT; - struct extent_buffer *eb; - struct extent_buffer *exists = NULL; - struct page *p; - struct address_space *mapping = fs_info->btree_inode->i_mapping; - int uptodate = 1; - int ret; - - if (!IS_ALIGNED(start, fs_info->sectorsize)) { - btrfs_err(fs_info, "bad tree block start %llu", start); - return ERR_PTR(-EINVAL); - } - - eb = find_extent_buffer(fs_info, start); - if (eb) - return eb; - - eb = __alloc_extent_buffer(fs_info, start, len); - if (!eb) - return ERR_PTR(-ENOMEM); - - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++, index++) { - p = find_or_create_page(mapping, index, GFP_NOFS|__GFP_NOFAIL); - if (!p) { - exists = ERR_PTR(-ENOMEM); - goto free_eb; - } - - spin_lock(&mapping->private_lock); - if (PagePrivate(p)) { - /* - * We could have already allocated an eb for this page - * and attached one so lets see if we can get a ref on - * the existing eb, and if we can we know it's good and - * we can just return that one, else we know we can just - * overwrite page->private. - */ - exists = (struct extent_buffer *)p->private; - if (atomic_inc_not_zero(&exists->refs)) { - spin_unlock(&mapping->private_lock); - unlock_page(p); - put_page(p); - mark_extent_buffer_accessed(exists, p); - goto free_eb; - } - exists = NULL; - - /* - * Do this so attach doesn't complain and we need to - * drop the ref the old guy had. - */ - ClearPagePrivate(p); - WARN_ON(PageDirty(p)); - put_page(p); - } - attach_extent_buffer_page(eb, p); - spin_unlock(&mapping->private_lock); - WARN_ON(PageDirty(p)); - eb->pages[i] = p; - if (!PageUptodate(p)) - uptodate = 0; - - /* - * We can't unlock the pages just yet since the extent buffer - * hasn't been properly inserted in the radix tree, this - * opens a race with btree_releasepage which can free a page - * while we are still filling in all pages for the buffer and - * we could crash. - */ - } - if (uptodate) - set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); -again: - ret = radix_tree_preload(GFP_NOFS); - if (ret) { - exists = ERR_PTR(ret); - goto free_eb; - } - - spin_lock(&fs_info->buffer_lock); - ret = radix_tree_insert(&fs_info->buffer_radix, - start >> PAGE_SHIFT, eb); - spin_unlock(&fs_info->buffer_lock); - radix_tree_preload_end(); - if (ret == -EEXIST) { - exists = find_extent_buffer(fs_info, start); - if (exists) - goto free_eb; - else - goto again; - } - /* add one reference for the tree */ - check_buffer_tree_ref(eb); - set_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags); - - /* - * Now it's safe to unlock the pages because any calls to - * btree_releasepage will correctly detect that a page belongs to a - * live buffer and won't free them prematurely. - */ - for (i = 0; i < num_pages; i++) - unlock_page(eb->pages[i]); - return eb; - -free_eb: - WARN_ON(!atomic_dec_and_test(&eb->refs)); - for (i = 0; i < num_pages; i++) { - if (eb->pages[i]) - unlock_page(eb->pages[i]); - } - - btrfs_release_extent_buffer(eb); - return exists; -} - -static inline void btrfs_release_extent_buffer_rcu(struct rcu_head *head) -{ - struct extent_buffer *eb = - container_of(head, struct extent_buffer, rcu_head); - - __free_extent_buffer(eb); -} - -static int release_extent_buffer(struct extent_buffer *eb) -{ - lockdep_assert_held(&eb->refs_lock); - - WARN_ON(atomic_read(&eb->refs) == 0); - if (atomic_dec_and_test(&eb->refs)) { - if (test_and_clear_bit(EXTENT_BUFFER_IN_TREE, &eb->bflags)) { - struct btrfs_fs_info *fs_info = eb->fs_info; - - spin_unlock(&eb->refs_lock); - - spin_lock(&fs_info->buffer_lock); - radix_tree_delete(&fs_info->buffer_radix, - eb->start >> PAGE_SHIFT); - spin_unlock(&fs_info->buffer_lock); - } else { - spin_unlock(&eb->refs_lock); - } - - /* Should be safe to release our pages at this point */ - btrfs_release_extent_buffer_pages(eb); -#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS - if (unlikely(test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags))) { - __free_extent_buffer(eb); - return 1; - } -#endif - call_rcu(&eb->rcu_head, btrfs_release_extent_buffer_rcu); - return 1; - } - spin_unlock(&eb->refs_lock); - - return 0; -} - -void free_extent_buffer(struct extent_buffer *eb) -{ - int refs; - int old; - if (!eb) - return; - - while (1) { - refs = atomic_read(&eb->refs); - if ((!test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags) && refs <= 3) - || (test_bit(EXTENT_BUFFER_UNMAPPED, &eb->bflags) && - refs == 1)) - break; - old = atomic_cmpxchg(&eb->refs, refs, refs - 1); - if (old == refs) - return; - } - - spin_lock(&eb->refs_lock); - if (atomic_read(&eb->refs) == 2 && - test_bit(EXTENT_BUFFER_STALE, &eb->bflags) && - !extent_buffer_under_io(eb) && - test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) - atomic_dec(&eb->refs); - - /* - * I know this is terrible, but it's temporary until we stop tracking - * the uptodate bits and such for the extent buffers. - */ - release_extent_buffer(eb); -} - -void free_extent_buffer_stale(struct extent_buffer *eb) -{ - if (!eb) - return; - - spin_lock(&eb->refs_lock); - set_bit(EXTENT_BUFFER_STALE, &eb->bflags); - - if (atomic_read(&eb->refs) == 2 && !extent_buffer_under_io(eb) && - test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) - atomic_dec(&eb->refs); - release_extent_buffer(eb); -} - -void clear_extent_buffer_dirty(struct extent_buffer *eb) -{ - int i; - int num_pages; - struct page *page; - - num_pages = num_extent_pages(eb); - - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - if (!PageDirty(page)) - continue; - - lock_page(page); - WARN_ON(!PagePrivate(page)); - - clear_page_dirty_for_io(page); - xa_lock_irq(&page->mapping->i_pages); - if (!PageDirty(page)) - __xa_clear_mark(&page->mapping->i_pages, - page_index(page), PAGECACHE_TAG_DIRTY); - xa_unlock_irq(&page->mapping->i_pages); - ClearPageError(page); - unlock_page(page); - } - WARN_ON(atomic_read(&eb->refs) == 0); -} - -bool set_extent_buffer_dirty(struct extent_buffer *eb) -{ - int i; - int num_pages; - bool was_dirty; - - check_buffer_tree_ref(eb); - - was_dirty = test_and_set_bit(EXTENT_BUFFER_DIRTY, &eb->bflags); - - num_pages = num_extent_pages(eb); - WARN_ON(atomic_read(&eb->refs) == 0); - WARN_ON(!test_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)); - - if (!was_dirty) - for (i = 0; i < num_pages; i++) - set_page_dirty(eb->pages[i]); - -#ifdef CONFIG_BTRFS_DEBUG - for (i = 0; i < num_pages; i++) - ASSERT(PageDirty(eb->pages[i])); -#endif - - return was_dirty; -} - -void clear_extent_buffer_uptodate(struct extent_buffer *eb) -{ - int i; - struct page *page; - int num_pages; - - clear_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - if (page) - ClearPageUptodate(page); - } -} - -void set_extent_buffer_uptodate(struct extent_buffer *eb) -{ - int i; - struct page *page; - int num_pages; - - set_bit(EXTENT_BUFFER_UPTODATE, &eb->bflags); - num_pages = num_extent_pages(eb); - for (i = 0; i < num_pages; i++) { - page = eb->pages[i]; - SetPageUptodate(page); - } -} - int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num) { int i; @@ -3571,578 +2891,3 @@ int read_extent_buffer_pages(struct extent_buffer *eb, int wait, int mirror_num) } return ret; } - -void read_extent_buffer(const struct extent_buffer *eb, void *dstv, - unsigned long start, unsigned long len) -{ - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - char *dst = (char *)dstv; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - - if (start + len > eb->len) { - WARN(1, KERN_ERR "btrfs bad mapping eb start %llu len %lu, wanted %lu %lu\n", - eb->start, eb->len, start, len); - memset(dst, 0, len); - return; - } - - offset = offset_in_page(start_offset + start); - - while (len > 0) { - page = eb->pages[i]; - - cur = min(len, (PAGE_SIZE - offset)); - kaddr = page_address(page); - memcpy(dst, kaddr + offset, cur); - - dst += cur; - len -= cur; - offset = 0; - i++; - } -} - -int read_extent_buffer_to_user(const struct extent_buffer *eb, - void __user *dstv, - unsigned long start, unsigned long len) -{ - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - char __user *dst = (char __user *)dstv; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - int ret = 0; - - WARN_ON(start > eb->len); - WARN_ON(start + len > eb->start + eb->len); - - offset = offset_in_page(start_offset + start); - - while (len > 0) { - page = eb->pages[i]; - - cur = min(len, (PAGE_SIZE - offset)); - kaddr = page_address(page); - if (copy_to_user(dst, kaddr + offset, cur)) { - ret = -EFAULT; - break; - } - - dst += cur; - len -= cur; - offset = 0; - i++; - } - - return ret; -} - -/* - * return 0 if the item is found within a page. - * return 1 if the item spans two pages. - * return -EINVAL otherwise. - */ -int map_private_extent_buffer(const struct extent_buffer *eb, - unsigned long start, unsigned long min_len, - char **map, unsigned long *map_start, - unsigned long *map_len) -{ - size_t offset; - char *kaddr; - struct page *p; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - unsigned long end_i = (start_offset + start + min_len - 1) >> - PAGE_SHIFT; - - if (start + min_len > eb->len) { - WARN(1, KERN_ERR "btrfs bad mapping eb start %llu len %lu, wanted %lu %lu\n", - eb->start, eb->len, start, min_len); - return -EINVAL; - } - - if (i != end_i) - return 1; - - if (i == 0) { - offset = start_offset; - *map_start = 0; - } else { - offset = 0; - *map_start = ((u64)i << PAGE_SHIFT) - start_offset; - } - - p = eb->pages[i]; - kaddr = page_address(p); - *map = kaddr + offset; - *map_len = PAGE_SIZE - offset; - return 0; -} - -int memcmp_extent_buffer(const struct extent_buffer *eb, const void *ptrv, - unsigned long start, unsigned long len) -{ - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - char *ptr = (char *)ptrv; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - int ret = 0; - - WARN_ON(start > eb->len); - WARN_ON(start + len > eb->start + eb->len); - - offset = offset_in_page(start_offset + start); - - while (len > 0) { - page = eb->pages[i]; - - cur = min(len, (PAGE_SIZE - offset)); - - kaddr = page_address(page); - ret = memcmp(ptr, kaddr + offset, cur); - if (ret) - break; - - ptr += cur; - len -= cur; - offset = 0; - i++; - } - return ret; -} - -void write_extent_buffer_chunk_tree_uuid(struct extent_buffer *eb, - const void *srcv) -{ - char *kaddr; - - WARN_ON(!PageUptodate(eb->pages[0])); - kaddr = page_address(eb->pages[0]); - memcpy(kaddr + offsetof(struct btrfs_header, chunk_tree_uuid), srcv, - BTRFS_FSID_SIZE); -} - -void write_extent_buffer_fsid(struct extent_buffer *eb, const void *srcv) -{ - char *kaddr; - - WARN_ON(!PageUptodate(eb->pages[0])); - kaddr = page_address(eb->pages[0]); - memcpy(kaddr + offsetof(struct btrfs_header, fsid), srcv, - BTRFS_FSID_SIZE); -} - -void write_extent_buffer(struct extent_buffer *eb, const void *srcv, - unsigned long start, unsigned long len) -{ - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - char *src = (char *)srcv; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - - WARN_ON(start > eb->len); - WARN_ON(start + len > eb->start + eb->len); - - offset = offset_in_page(start_offset + start); - - while (len > 0) { - page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); - - cur = min(len, PAGE_SIZE - offset); - kaddr = page_address(page); - memcpy(kaddr + offset, src, cur); - - src += cur; - len -= cur; - offset = 0; - i++; - } -} - -void memzero_extent_buffer(struct extent_buffer *eb, unsigned long start, - unsigned long len) -{ - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - size_t start_offset = offset_in_page(eb->start); - unsigned long i = (start_offset + start) >> PAGE_SHIFT; - - WARN_ON(start > eb->len); - WARN_ON(start + len > eb->start + eb->len); - - offset = offset_in_page(start_offset + start); - - while (len > 0) { - page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); - - cur = min(len, PAGE_SIZE - offset); - kaddr = page_address(page); - memset(kaddr + offset, 0, cur); - - len -= cur; - offset = 0; - i++; - } -} - -void copy_extent_buffer_full(struct extent_buffer *dst, - struct extent_buffer *src) -{ - int i; - int num_pages; - - ASSERT(dst->len == src->len); - - num_pages = num_extent_pages(dst); - for (i = 0; i < num_pages; i++) - copy_page(page_address(dst->pages[i]), - page_address(src->pages[i])); -} - -void copy_extent_buffer(struct extent_buffer *dst, struct extent_buffer *src, - unsigned long dst_offset, unsigned long src_offset, - unsigned long len) -{ - u64 dst_len = dst->len; - size_t cur; - size_t offset; - struct page *page; - char *kaddr; - size_t start_offset = offset_in_page(dst->start); - unsigned long i = (start_offset + dst_offset) >> PAGE_SHIFT; - - WARN_ON(src->len != dst_len); - - offset = offset_in_page(start_offset + dst_offset); - - while (len > 0) { - page = dst->pages[i]; - WARN_ON(!PageUptodate(page)); - - cur = min(len, (unsigned long)(PAGE_SIZE - offset)); - - kaddr = page_address(page); - read_extent_buffer(src, kaddr + offset, src_offset, cur); - - src_offset += cur; - len -= cur; - offset = 0; - i++; - } -} - -/* - * eb_bitmap_offset() - calculate the page and offset of the byte containing the - * given bit number - * @eb: the extent buffer - * @start: offset of the bitmap item in the extent buffer - * @nr: bit number - * @page_index: return index of the page in the extent buffer that contains the - * given bit number - * @page_offset: return offset into the page given by page_index - * - * This helper hides the ugliness of finding the byte in an extent buffer which - * contains a given bit. - */ -static inline void eb_bitmap_offset(struct extent_buffer *eb, - unsigned long start, unsigned long nr, - unsigned long *page_index, - size_t *page_offset) -{ - size_t start_offset = offset_in_page(eb->start); - size_t byte_offset = BIT_BYTE(nr); - size_t offset; - - /* - * The byte we want is the offset of the extent buffer + the offset of - * the bitmap item in the extent buffer + the offset of the byte in the - * bitmap item. - */ - offset = start_offset + start + byte_offset; - - *page_index = offset >> PAGE_SHIFT; - *page_offset = offset_in_page(offset); -} - -/** - * extent_buffer_test_bit - determine whether a bit in a bitmap item is set - * @eb: the extent buffer - * @start: offset of the bitmap item in the extent buffer - * @nr: bit number to test - */ -int extent_buffer_test_bit(struct extent_buffer *eb, unsigned long start, - unsigned long nr) -{ - u8 *kaddr; - struct page *page; - unsigned long i; - size_t offset; - - eb_bitmap_offset(eb, start, nr, &i, &offset); - page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); - kaddr = page_address(page); - return 1U & (kaddr[offset] >> (nr & (BITS_PER_BYTE - 1))); -} - -/** - * extent_buffer_bitmap_set - set an area of a bitmap - * @eb: the extent buffer - * @start: offset of the bitmap item in the extent buffer - * @pos: bit number of the first bit - * @len: number of bits to set - */ -void extent_buffer_bitmap_set(struct extent_buffer *eb, unsigned long start, - unsigned long pos, unsigned long len) -{ - u8 *kaddr; - struct page *page; - unsigned long i; - size_t offset; - const unsigned int size = pos + len; - int bits_to_set = BITS_PER_BYTE - (pos % BITS_PER_BYTE); - u8 mask_to_set = BITMAP_FIRST_BYTE_MASK(pos); - - eb_bitmap_offset(eb, start, pos, &i, &offset); - page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); - kaddr = page_address(page); - - while (len >= bits_to_set) { - kaddr[offset] |= mask_to_set; - len -= bits_to_set; - bits_to_set = BITS_PER_BYTE; - mask_to_set = ~0; - if (++offset >= PAGE_SIZE && len > 0) { - offset = 0; - page = eb->pages[++i]; - WARN_ON(!PageUptodate(page)); - kaddr = page_address(page); - } - } - if (len) { - mask_to_set &= BITMAP_LAST_BYTE_MASK(size); - kaddr[offset] |= mask_to_set; - } -} - - -/** - * extent_buffer_bitmap_clear - clear an area of a bitmap - * @eb: the extent buffer - * @start: offset of the bitmap item in the extent buffer - * @pos: bit number of the first bit - * @len: number of bits to clear - */ -void extent_buffer_bitmap_clear(struct extent_buffer *eb, unsigned long start, - unsigned long pos, unsigned long len) -{ - u8 *kaddr; - struct page *page; - unsigned long i; - size_t offset; - const unsigned int size = pos + len; - int bits_to_clear = BITS_PER_BYTE - (pos % BITS_PER_BYTE); - u8 mask_to_clear = BITMAP_FIRST_BYTE_MASK(pos); - - eb_bitmap_offset(eb, start, pos, &i, &offset); - page = eb->pages[i]; - WARN_ON(!PageUptodate(page)); - kaddr = page_address(page); - - while (len >= bits_to_clear) { - kaddr[offset] &= ~mask_to_clear; - len -= bits_to_clear; - bits_to_clear = BITS_PER_BYTE; - mask_to_clear = ~0; - if (++offset >= PAGE_SIZE && len > 0) { - offset = 0; - page = eb->pages[++i]; - WARN_ON(!PageUptodate(page)); - kaddr = page_address(page); - } - } - if (len) { - mask_to_clear &= BITMAP_LAST_BYTE_MASK(size); - kaddr[offset] &= ~mask_to_clear; - } -} - -static inline bool areas_overlap(unsigned long src, unsigned long dst, unsigned long len) -{ - unsigned long distance = (src > dst) ? src - dst : dst - src; - return distance < len; -} - -static void copy_pages(struct page *dst_page, struct page *src_page, - unsigned long dst_off, unsigned long src_off, - unsigned long len) -{ - char *dst_kaddr = page_address(dst_page); - char *src_kaddr; - int must_memmove = 0; - - if (dst_page != src_page) { - src_kaddr = page_address(src_page); - } else { - src_kaddr = dst_kaddr; - if (areas_overlap(src_off, dst_off, len)) - must_memmove = 1; - } - - if (must_memmove) - memmove(dst_kaddr + dst_off, src_kaddr + src_off, len); - else - memcpy(dst_kaddr + dst_off, src_kaddr + src_off, len); -} - -void memcpy_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, - unsigned long src_offset, unsigned long len) -{ - struct btrfs_fs_info *fs_info = dst->fs_info; - size_t cur; - size_t dst_off_in_page; - size_t src_off_in_page; - size_t start_offset = offset_in_page(dst->start); - unsigned long dst_i; - unsigned long src_i; - - if (src_offset + len > dst->len) { - btrfs_err(fs_info, - "memmove bogus src_offset %lu move len %lu dst len %lu", - src_offset, len, dst->len); - BUG(); - } - if (dst_offset + len > dst->len) { - btrfs_err(fs_info, - "memmove bogus dst_offset %lu move len %lu dst len %lu", - dst_offset, len, dst->len); - BUG(); - } - - while (len > 0) { - dst_off_in_page = offset_in_page(start_offset + dst_offset); - src_off_in_page = offset_in_page(start_offset + src_offset); - - dst_i = (start_offset + dst_offset) >> PAGE_SHIFT; - src_i = (start_offset + src_offset) >> PAGE_SHIFT; - - cur = min(len, (unsigned long)(PAGE_SIZE - - src_off_in_page)); - cur = min_t(unsigned long, cur, - (unsigned long)(PAGE_SIZE - dst_off_in_page)); - - copy_pages(dst->pages[dst_i], dst->pages[src_i], - dst_off_in_page, src_off_in_page, cur); - - src_offset += cur; - dst_offset += cur; - len -= cur; - } -} - -void memmove_extent_buffer(struct extent_buffer *dst, unsigned long dst_offset, - unsigned long src_offset, unsigned long len) -{ - struct btrfs_fs_info *fs_info = dst->fs_info; - size_t cur; - size_t dst_off_in_page; - size_t src_off_in_page; - unsigned long dst_end = dst_offset + len - 1; - unsigned long src_end = src_offset + len - 1; - size_t start_offset = offset_in_page(dst->start); - unsigned long dst_i; - unsigned long src_i; - - if (src_offset + len > dst->len) { - btrfs_err(fs_info, - "memmove bogus src_offset %lu move len %lu len %lu", - src_offset, len, dst->len); - BUG(); - } - if (dst_offset + len > dst->len) { - btrfs_err(fs_info, - "memmove bogus dst_offset %lu move len %lu len %lu", - dst_offset, len, dst->len); - BUG(); - } - if (dst_offset < src_offset) { - memcpy_extent_buffer(dst, dst_offset, src_offset, len); - return; - } - while (len > 0) { - dst_i = (start_offset + dst_end) >> PAGE_SHIFT; - src_i = (start_offset + src_end) >> PAGE_SHIFT; - - dst_off_in_page = offset_in_page(start_offset + dst_end); - src_off_in_page = offset_in_page(start_offset + src_end); - - cur = min_t(unsigned long, len, src_off_in_page + 1); - cur = min(cur, dst_off_in_page + 1); - copy_pages(dst->pages[dst_i], dst->pages[src_i], - dst_off_in_page - cur + 1, - src_off_in_page - cur + 1, cur); - - dst_end -= cur; - src_end -= cur; - len -= cur; - } -} - -int try_release_extent_buffer(struct page *page) -{ - struct extent_buffer *eb; - - /* - * We need to make sure nobody is attaching this page to an eb right - * now. - */ - spin_lock(&page->mapping->private_lock); - if (!PagePrivate(page)) { - spin_unlock(&page->mapping->private_lock); - return 1; - } - - eb = (struct extent_buffer *)page->private; - BUG_ON(!eb); - - /* - * This is a little awful but should be ok, we need to make sure that - * the eb doesn't disappear out from under us while we're looking at - * this page. - */ - spin_lock(&eb->refs_lock); - if (atomic_read(&eb->refs) != 1 || extent_buffer_under_io(eb)) { - spin_unlock(&eb->refs_lock); - spin_unlock(&page->mapping->private_lock); - return 0; - } - spin_unlock(&page->mapping->private_lock); - - /* - * If tree ref isn't set then we know the ref on this eb is a real ref, - * so just return, this page will likely be freed soon anyway. - */ - if (!test_and_clear_bit(EXTENT_BUFFER_TREE_REF, &eb->bflags)) { - spin_unlock(&eb->refs_lock); - return 0; - } - - return release_extent_buffer(eb); -}