From patchwork Fri Jan 25 15:42:54 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liu Bo X-Patchwork-Id: 2045991 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 3C7ECE0175 for ; Fri, 25 Jan 2013 15:46:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755877Ab3AYPp6 (ORCPT ); Fri, 25 Jan 2013 10:45:58 -0500 Received: from userp1040.oracle.com ([156.151.31.81]:36994 "EHLO userp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755629Ab3AYPp5 (ORCPT ); Fri, 25 Jan 2013 10:45:57 -0500 Received: from acsinet21.oracle.com (acsinet21.oracle.com [141.146.126.237]) by userp1040.oracle.com (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id r0PFjbw0016660 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 25 Jan 2013 15:45:37 GMT Received: from acsmt357.oracle.com (acsmt357.oracle.com [141.146.40.157]) by acsinet21.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id r0PFjaRa000792 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 25 Jan 2013 15:45:36 GMT Received: from abhmt116.oracle.com (abhmt116.oracle.com [141.146.116.68]) by acsmt357.oracle.com (8.12.11.20060308/8.12.11) with ESMTP id r0PFjaBe022879; Fri, 25 Jan 2013 09:45:36 -0600 Received: from liubo (/219.144.142.233) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 25 Jan 2013 07:45:34 -0800 Date: Fri, 25 Jan 2013 23:42:54 +0800 From: Liu Bo To: Mitch Harder Cc: linux-btrfs@vger.kernel.org, chris.mason@fusionio.com, JBacik@fusionio.com, dave@jikos.cz, kitayama@cl.bb4u.ne.jp, miaox@cn.fujitsu.com Subject: Re: [PATCH V5] Btrfs: snapshot-aware defrag Message-ID: <20130125154236.GA3997@liubo> Reply-To: bo.li.liu@oracle.com References: <1358339768-2314-1-git-send-email-bo.li.liu@oracle.com> <20130123075155.GE17162@liubo.jp.oracle.com> <20130124005221.GA28406@liubo> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: acsinet21.oracle.com [141.146.126.237] Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Fri, Jan 25, 2013 at 08:55:58AM -0600, Mitch Harder wrote: > On Wed, Jan 23, 2013 at 6:52 PM, Liu Bo wrote: > > On Wed, Jan 23, 2013 at 10:05:04AM -0600, Mitch Harder wrote: > >> On Wed, Jan 23, 2013 at 1:51 AM, Liu Bo wrote: > >> > On Tue, Jan 22, 2013 at 11:41:19AM -0600, Mitch Harder wrote: > >> >> On Thu, Jan 17, 2013 at 8:42 AM, Mitch Harder > >> >> wrote: > >> >> > On Wed, Jan 16, 2013 at 6:36 AM, Liu Bo wrote: > >> >> >> This comes from one of btrfs's project ideas, > >> >> >> As we defragment files, we break any sharing from other snapshots. > >> >> >> The balancing code will preserve the sharing, and defrag needs to grow this > >> >> >> as well. > >> > [...] > >> >> > > >> >> > I've been testing this patch on a 3.7.2 kernel merged with the > >> >> > for-linus branch for the 3.8_rc kernels, and I'm seeing the following > >> >> > error: > >> >> > > >> >> > >> >> I've reproduced the error with CONFIG_DEBUG_LIST enabled, which shows > >> >> some problem with an entry in the list. > >> >> > >> >> [59312.260441] ------------[ cut here ]------------ > >> >> [59312.260454] WARNING: at lib/list_debug.c:62 __list_del_entry+0x8d/0x98() > >> >> [59312.260458] Hardware name: OptiPlex 745 > >> >> [59312.260461] list_del corruption. next->prev should be > >> >> ffff88006511c438, but was dead000000200200 > >> > > >> > LIST_POISON2 -> (000000200200) > >> > So we can know that the next one is deleted from the list even _earlier_ > >> > than the current one is. > >> > > >> > Any other messages before this warning complains? > >> > > >> > >> Just some normal feedback from a metadata balance I had run. > > > > Well, these do fit my expectation, since balance also involves with playing with > > root_list, which may lead to the bad situation. > > > >> > >> [14057.193343] device fsid 28c688c5-7dbd-4071-b271-1bf6726d8835 devid > >> 1 transid 4 /dev/sda7 > >> [14057.194438] btrfs: force lzo compression > >> [14057.194446] btrfs: enabling auto defrag > >> [14057.194449] btrfs: disk space caching is enabled > >> [14057.194452] btrfs flagging fs with big metadata feature > >> [14057.194455] btrfs: lzo incompat flag set. > >> [57508.799193] btrfs: relocating block group 14516486144 flags 4 > >> [57632.178797] btrfs: found 6775 extents > >> [57633.214701] btrfs: relocating block group 11832131584 flags 4 > >> [57776.400102] btrfs: found 6480 extents > >> [57777.021175] btrfs: relocating block group 10489954304 flags 4 > >> [57949.182725] btrfs: found 6681 extents > >> [59312.260441] ------------[ cut here ]------------ > >> [59312.260454] WARNING: at lib/list_debug.c:62 __list_del_entry+0x8d/0x98() > >> [59312.260458] Hardware name: OptiPlex 745 > >> ... > >> > >> I'm going to try to wrap some debugging around the section of code in > >> btrfs_clean_old_snapshots() where the dead_roots list is spliced onto > >> the root list being processed. The double entry may be slipping in > >> here. > >> > >> 1764 spin_lock(&fs_info->trans_lock); > >> 1765 list_splice_init(&fs_info->dead_roots, &list); > >> 1766 spin_unlock(&fs_info->trans_lock); > > > > hmm, I don't think there is anything wrong in this code. But you can > > give it a shot anyway :) > > > > I've changed up my reproducer to try some things that may hit the > issue quicker and more reliably. > > It gave me a slightly different set of warnings in dmesg, which seem > to suggest issues in the dead_root list. Great! Many thanks for nail it down, we really shouldn't iput() after btrfs_iget(). Could you please try this(remove iput()) and see if it gets us rid of the trouble? thanks, liubo > > [43925.656065] device fsid a8f6fadb-3022-4c01-b369-f1f3f638c052 devid > 1 transid 310 /dev/sda7 > [43925.658062] btrfs: force lzo compression > [43925.658072] btrfs: enabling auto defrag > [43925.658075] btrfs: disk space caching is enabled > [43925.658078] btrfs: lzo incompat flag set. > [44503.421293] btrfs: unlinked 1 orphans > [44898.287365] btrfs: unlinked 1 orphans > [45080.641383] btrfs: unlinked 1 orphans > [45250.063773] btrfs: unlinked 1 orphans > [46223.387355] btrfs: unlinked 1 orphans > [46476.473944] btrfs: unlinked 1 orphans > [46499.665615] btrfs: unlinked 1 orphans > [46769.785454] ------------[ cut here ]------------ > [46769.785471] WARNING: at lib/list_debug.c:36 __list_add+0x9d/0xba() > [46769.785474] Hardware name: OptiPlex 745 > [46769.785478] list_add double add: new=ffff880050c27c38, > prev=ffff880078f3e720, next=ffff880050c27c38. > [46769.785480] Modules linked in: ipv6 snd_hda_codec_analog > snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_timer > tg3 sr_mod snd i2c_i801 ppdev parport_pc iTCO_wdt iTCO_vendor_support > lpc_ich pcspkr parport floppy serio_raw microcode ablk_helper cryptd > lrw xts gf128mul aes_x86_64 sha256_generic fuse xfs nfs lockd sunrpc > reiserfs btrfs zlib_deflate ext4 jbd2 ext3 jbd ext2 mbcache sl811_hcd > hid_generic xhci_hcd ohci_hcd uhci_hcd ehci_hcd > [46769.785537] Pid: 18291, comm: btrfs-endio-wri Not tainted 3.7.4-sad-v1+ #3 > [46769.785539] Call Trace: > [46769.785549] [] warn_slowpath_common+0x83/0x9b > [46769.785553] [] warn_slowpath_fmt+0x46/0x48 > [46769.785558] [] __list_add+0x9d/0xba > [46769.785586] [] btrfs_add_dead_root+0x42/0x56 [btrfs] > [46769.785603] [] btrfs_destroy_inode+0x227/0x25b [btrfs] > [46769.785611] [] destroy_inode+0x3b/0x54 > [46769.785615] [] evict+0x149/0x151 > [46769.785619] [] iput+0x12c/0x135 > [46769.785636] [] relink_extent_backref+0x669/0x6af [btrfs] > [46769.785642] [] ? __slab_free+0x17c/0x21b > [46769.785658] [] ? > btrfs_finish_ordered_io+0x770/0x827 [btrfs] > [46769.785674] [] btrfs_finish_ordered_io+0x740/0x827 [btrfs] > [46769.785691] [] finish_ordered_fn+0x15/0x17 [btrfs] > [46769.785706] [] worker_loop+0x14c/0x493 [btrfs] > [46769.785722] [] ? btrfs_queue_worker+0x258/0x258 [btrfs] > [46769.785728] [] kthread+0xba/0xc2 > [46769.785732] [] ? kthread_freezable_should_stop+0x52/0x52 > [46769.785737] [] ret_from_fork+0x7c/0xb0 > [46769.785741] [] ? kthread_freezable_should_stop+0x52/0x52 > [46769.785745] ---[ end trace 7528086f91b151b5 ]--- > [46799.053062] ------------[ cut here ]------------ > [46799.053078] WARNING: at lib/list_debug.c:62 __list_del_entry+0x8d/0x98() > [46799.053082] Hardware name: OptiPlex 745 > [46799.053087] list_del corruption. next->prev should be > ffff880050c27c38, but was ffff8800057fde38 > [46799.053090] Modules linked in: ipv6 snd_hda_codec_analog > snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_timer > tg3 sr_mod snd i2c_i801 ppdev parport_pc iTCO_wdt iTCO_vendor_support > lpc_ich pcspkr parport floppy serio_raw microcode ablk_helper cryptd > lrw xts gf128mul aes_x86_64 sha256_generic fuse xfs nfs lockd sunrpc > reiserfs btrfs zlib_deflate ext4 jbd2 ext3 jbd ext2 mbcache sl811_hcd > hid_generic xhci_hcd ohci_hcd uhci_hcd ehci_hcd > [46799.053163] Pid: 18210, comm: btrfs-cleaner Tainted: G W > 3.7.4-sad-v1+ #3 > [46799.053166] Call Trace: > [46799.053180] [] warn_slowpath_common+0x83/0x9b > [46799.053184] [] warn_slowpath_fmt+0x46/0x48 > [46799.053190] [] ? __trace_bprintk+0x48/0x4a > [46799.053194] [] __list_del_entry+0x8d/0x98 > [46799.053198] [] list_del+0xe/0x2e > [46799.053220] [] > btrfs_clean_old_snapshots+0xed/0x150 [btrfs] > [46799.053235] [] cleaner_kthread+0x5a/0xe6 [btrfs] > [46799.053249] [] ? transaction_kthread+0x1a0/0x1a0 [btrfs] > [46799.053254] [] kthread+0xba/0xc2 > [46799.053259] [] ? kthread_freezable_should_stop+0x52/0x52 > [46799.053264] [] ret_from_fork+0x7c/0xb0 > [46799.053269] [] ? kthread_freezable_should_stop+0x52/0x52 > [46799.053272] ---[ end trace 7528086f91b151b6 ]--- > [46811.162649] ------------[ cut here ]------------ > [46811.162665] WARNING: at lib/list_debug.c:53 __list_del_entry+0x8d/0x98() > [46811.162669] Hardware name: OptiPlex 745 > [46811.162674] list_del corruption, ffff880050c27c38->next is > LIST_POISON1 (dead000000100100) > [46811.162678] Modules linked in: ipv6 snd_hda_codec_analog > snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_timer > tg3 sr_mod snd i2c_i801 ppdev parport_pc iTCO_wdt iTCO_vendor_support > lpc_ich pcspkr parport floppy serio_raw microcode ablk_helper cryptd > lrw xts gf128mul aes_x86_64 sha256_generic fuse xfs nfs lockd sunrpc > reiserfs btrfs zlib_deflate ext4 jbd2 ext3 jbd ext2 mbcache sl811_hcd > hid_generic xhci_hcd ohci_hcd uhci_hcd ehci_hcd > [46811.162750] Pid: 18210, comm: btrfs-cleaner Tainted: G W > 3.7.4-sad-v1+ #3 > [46811.162754] Call Trace: > [46811.162764] [] warn_slowpath_common+0x83/0x9b > [46811.162771] [] warn_slowpath_fmt+0x46/0x48 > [46811.162779] [] ? __trace_bprintk+0x48/0x4a > [46811.162785] [] __list_del_entry+0x8d/0x98 > [46811.162791] [] list_del+0xe/0x2e > [46811.162820] [] > btrfs_clean_old_snapshots+0xed/0x150 [btrfs] > [46811.162841] [] cleaner_kthread+0x5a/0xe6 [btrfs] > [46811.162862] [] ? transaction_kthread+0x1a0/0x1a0 [btrfs] > [46811.162869] [] kthread+0xba/0xc2 > [46811.162875] [] ? kthread_freezable_should_stop+0x52/0x52 > [46811.162882] [] ret_from_fork+0x7c/0xb0 > [46811.162888] [] ? kthread_freezable_should_stop+0x52/0x52 > [46811.162892] ---[ end trace 7528086f91b151b7 ]--- > [46811.162904] BUG: unable to handle kernel paging request at 0000000047c5a000 > [46811.163003] IP: [] > btrfs_clean_old_snapshots+0x103/0x150 [btrfs] > [46811.163003] PGD 0 > [46811.163003] Oops: 0000 [#1] SMP > [46811.163003] Modules linked in: ipv6 snd_hda_codec_analog > snd_hda_intel snd_hda_codec snd_hwdep snd_pcm snd_page_alloc snd_timer > tg3 sr_mod snd i2c_i801 ppdev parport_pc iTCO_wdt iTCO_vendor_support > lpc_ich pcspkr parport floppy serio_raw microcode ablk_helper cryptd > lrw xts gf128mul aes_x86_64 sha256_generic fuse xfs nfs lockd sunrpc > reiserfs btrfs zlib_deflate ext4 jbd2 ext3 jbd ext2 mbcache sl811_hcd > hid_generic xhci_hcd ohci_hcd uhci_hcd ehci_hcd > [46811.163003] CPU 0 > [46811.163003] Pid: 18210, comm: btrfs-cleaner Tainted: G W > 3.7.4-sad-v1+ #3 Dell Inc. OptiPlex 745 > /0WF810 > [46811.163003] RIP: 0010:[] [] > btrfs_clean_old_snapshots+0x103/0x150 [btrfs] > [46811.163003] RSP: 0018:ffff8800057fde38 EFLAGS: 00010296 > [46811.163003] RAX: 0000000047c5a000 RBX: ffff880050c27800 RCX: 0000000000000008 > [46811.163003] RDX: 0000000000000000 RSI: ffff8800057fdd70 RDI: ffff880050c27c70 > [46811.163003] RBP: ffff8800057fde78 R08: 0000000000000000 R09: 0000000000000283 > [46811.163003] R10: 0000000000000001 R11: 0000000000000000 R12: ffff880000000000 > [46811.163003] R13: 0000160000000000 R14: ffff8800057fde38 R15: ffff880050c27c38 > [46811.163003] FS: 0000000000000000(0000) GS:ffff88007f200000(0000) > knlGS:0000000000000000 > [46811.163003] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b > [46811.163003] CR2: 0000000047c5a000 CR3: 000000003f270000 CR4: 00000000000007f0 > [46811.163003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > [46811.163003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > [46811.163003] Process btrfs-cleaner (pid: 18210, threadinfo > ffff8800057fc000, task ffff88007c030d40) > [46811.163003] Stack: > [46811.163003] ffff880050c27c38 ffff88001f488438 ffff8800057fde48 > ffff88002d15b800 > [46811.163003] ffff88007c030d40 ffff88007c030d40 ffff88007c030d40 > 0000000000000000 > [46811.163003] ffff8800057fdeb8 ffffffffa0173d7d ffff88002d15b800 > 0000000000000000 > [46811.163003] Call Trace: > [46811.163003] [] cleaner_kthread+0x5a/0xe6 [btrfs] > [46811.163003] [] ? transaction_kthread+0x1a0/0x1a0 [btrfs] > [46811.163003] [] kthread+0xba/0xc2 > [46811.163003] [] ? kthread_freezable_should_stop+0x52/0x52 > [46811.163003] [] ret_from_fork+0x7c/0xb0 > [46811.163003] [] ? kthread_freezable_should_stop+0x52/0x52 > [46811.163003] Code: c7 c7 d5 b2 17 a0 31 c0 e8 b4 01 f3 e0 4c 89 ff > e8 bb e4 08 e1 48 89 df e8 f2 f5 03 00 49 8b 87 c8 fb ff ff 48 8b 80 > 50 01 00 00 <48> 8b 00 4c 01 e8 48 c1 f8 06 48 c1 e0 0c 42 0f b6 44 20 > 3f 31 > [46811.163003] RIP [] > btrfs_clean_old_snapshots+0x103/0x150 [btrfs] > [46811.163003] RSP > [46811.163003] CR2: 0000000047c5a000 > [46811.238512] ---[ end trace 7528086f91b151b8 ]--- --- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 1683f48..c7a0fb7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2337,7 +2337,6 @@ out_free_path: out_unlock: unlock_extent_cached(&BTRFS_I(inode)->io_tree, lock_start, lock_end, &cached, GFP_NOFS); - iput(inode); return ret; }